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SUMMARY 

This study investicated the offect of using different rofrac- 
tion models upon the second order astronomic latitudes obtained from 
observed stars, The three refraction models used wero those of Baldini, 
Garfinkel and the United States Coast and Geodetic Survey. It was 
found when using the method of observing a north and a south star to 
obtain the final latitnde, that no significant difference existed in 
the final combined values of latitude using any of tho refraction 
models. However, the difference betwoen the north star and south star 
values of latitude were slightly larger using the Garfinkel Model than 


when the other two models were used, 


ay, 
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CHAPTER 1 


PURPOSE AND SCOPE 

The major investigation undertaken in this thesis is the examie 
nation of the effect of the rofraction values given by three solected 
refraction models upon tho astronomical latitude detemnined from a sot 
of star observations at a given station. The throe refraction models 
which were used are thoso of the United States Coast and Geodetic 
Survey, (3) Baldini (1) and Garfinkel (5), The primary objective of 
the project has been to dotermine whether any significant difference in 
the final value of the astronomic latitude exists when each of the 
refraction models is used, This would also spetede any effect on the 
standard error of the mean latitude for each station, The next objec- 
tive of the study was to determine if any model was sufficiently 
accurate to enable the observer to obtain his latitude position by 
observing ono single star, either north or south, in place of the current 
method of observing both a north and south star, Finally, it was attempt- 
ed to suggest a mothod which would include an adjustment of the re- 
fraction coefficients from the observed data at a station, The Garfinkel 
model was tho model used for this adjustment project. 

The observed data for thirtoen second order latitude stations in 
Southwestern Ohio were used for this investigation. Each of these 
stations was reduced using each refraction model and tho results come 
pared, This is listed in tablo 4 The IBM 7094 electronic computer was 
used for these reductions and the programs wero written in the SCATRAN 
language, 

In addition to the actwal station reductions various aspects of 


the refraction models were investigated to determine the offocts caused 








by the variation of certain assumed values or constants upon the computed 
refraction, This was porformed on the Paldini model to detormine the 
effects of the relative humidity and the wave length of the obsorved 
light and on the Garfinkel modol for certain constants. The refraction 
values computed by each of the three models for various zenith distances 
were compared for a range of temperature and pressure combinations for 
selected conditions, 

This roport will be genorally divided into five main sections 
in addition to this one. The second section will contain some basic 
definitions and describe selected classical theories of the variation 
of the earth's atmosphere. Tho third section will briefly describe the 
basic assumptions that are made for the three refraction models and 
list their final computation equations, This section will also include 
the investigations porformed on the various models and the results of 
the comparison of their refraction values, The fourth section will 
deal with the latitude reduction methods used in this study and the 
results obtained. The fifth major section will be devoted to the 
adjustment procedures and results obtained using the station data and a 
form of the Garfinkel Refraction Equation, The final section will serve 
as @ summary and a statement of the major conclusions of the inves-= 


tigation, 





CHAPTER 2 


BAC KGROUND 

2.1 General 

For this paper, astronomic refraction will be defined as tho 
apparent displacement of an object that results from light rays from a 
source outsido the atmosphere being bent in passing through the atmos- 
phere, (6) The measure of this quantity is the chango produced in the 
direction of the light rays, The total amount of this change depends 
on the density of the various strata of air through which the light rays 
pass and upon the direction of these strata with respoct to the vertical. 
ne | 

It is the consideration of the constitution of the atmosphere 
which gives rise to the differences in the various proposed rofraction 
models, The atmosphere of tho earth is known to be more dense near the 
surface of the earth and gradually decreases in donsity to its upper 
limit, For our purposes the upper limit of the atmosphere will be tho 
point at which the density is so low that its offect on a ray of light 
is negligible, Since this condition causes the ray of light to always 
pass from a medium of lower donsity to one of higher donsity, the result 
will be that al] observed objects will appear to be higher above the 
horizon than they actually are. Figure 1 shows the path of a ray of 
light from the time it enters the atmosphere until it is seen by an 
observer. (2,6,7) 

The index of refraction for a given media of constant density 
shall be defined hero as the ratio of the sine of the angle of incidence 
to the sine of the angle of rofraction, when these anglos are measured 


with respect to a normal to the boundary surfaco where the light ray 
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Figure 1, Refraction Caused by the Earth's Atmosphere 





passes into the given madia from a vacuum, Tho anglo of incidonce is 
measured at the boundary surfaco before the ray passes into tho media 
and the angle of refraction is measured after it has passed through the 
boundary surface, 

2.2 Classical Theorios of tho Variation of the Earth's Atmosphere, 

The density variation of the carth's atmosphero from the surface 
to the upper limit is assumed to be subject to the conditions of 
equilibrium. Therefore, it can bo gonorally stated that the donsity 
varies with temperature and pressure, Since the astronomic refraction 
is dependent on the density, it may also bo related to the variation of 
the temperature and pressure of the atmosphere. The various rofraction 
tables which have been constructed for astronomical use rest upon 
different expressions for the variation of the density of the atmos- 
phere with the altitude, Four of the classical theories for this vari- 
ation will bo reviewed in the following paragraphs, 

The first real attempt to describe the variation of the atmos- 
phere was put forth by Nowton. This hypothesis assumed a constant 
temperature at all altitudes, Tho relation has been discarded in modern 
times and therefore this theory should be considered only in an histor-~ 
ical sense, (7) 

Bessel proposed that the density could be expressed in terms of 
altitude only and therefore assumed no law of temperature, The ratio of 


the densities at two altitudes is expressed as 
oh eo 
yy om 
whore Q is the density at altitudo h and the subscript ono rofers to the 
conditions of the base station, Kis a constant factor used for com-~ 


putations (K=0.9649). 62) 





Ivory put forth the hypothesis that the tenperaturo diminishos 
at a uniform rate with the heirfht at all altitudes, This ropresents 
the ratio of the donsitics as 
where 

T=T G-@h) and = a i : 
In this rolation hop is the altitude at which the atmosphere would terie 
nate, @ is a constant factor used to rolato the temporature (T) at a 
given altitude (h) to the temporature at the base station (Ty) and 
the exponent y is expressed in terms of the height of the baso station 
(hy) and RQ . 


jis h 
SAX \ = mie = pe 
hy OST, 


where the value of ¥ 41s a constant, doponding on the elasticity of air 
ata civen temperature and density, (7) 

The last classical hypothesis to bo givon horo states that tho 
temporature diminishes by a constant fraction of its absolute emount for 
every unit increase in altitude, This is in the form of a constant 
geomotrical progression which may be ropresented, in terms of the ratio 


of densities, as 


Ry 
where / VY = Ts ay (as » - BY 


The emall g represents the valuo of gravity and all the other terms havo 
been previously defined, (7) 

Thoso theories have been presonted as a background for the 
discussion of the spocific rofraction models with which wo are cone 


cerned in the next section, 





CHAPTER 3 


REFRACTION MODELS INVESTIGATSD FOR THIS PROJECT 
3.1 General 

The doscriptions of the throo refraction models which follow 
are not intended to be detailed discussions of the procedures used to 
derive the refraction equations, Rather, thoy simply attempt to give 
the basic assumptions upon which the models are based and the final 
forms of the rofraction equations used for computations, It will be 
attempted to point out any limitations introduced into the models and 
any similaritios which may exist. 

The computer programs used for the practical computations with 
each refraction model will be genorally described after the basic 
discussion of each model, The statements listings and required input 
data for each of the programs covered may be found in Appendix I, 

To save repetition with each refraction model, some basic terms 
common to all three models will be defined here, The quantity R will 
always represent the astronomic refraction, Z will be the zonith 
distance corrected for refraction and Zo will represent the observed 
zenith distance, T and P will represent tho absolute temperature and 
pressure respectively. When these last two quantities appear with the 
subscript (0) they will refer to the conditions at the observation 
station, All other quantities will be defined when they appear in the 
discussion for each model. 

3.2 The United States Coast and Goodetic Survey Refraction Model 

This refraction model is based on a theoretical background which 

4s combined with physical data to produce a proposed composition of the 


atmosphere, This model follows that proposed by Willis (11) and is 





generally based upon the theories of Kdwcomb.\?) 

First, the pressuro and temporaturo of the atmosphoro aro 
assumed to vary with respect to the density in the relation given for 
a perfect gas, This rolationship is given as 

e- 

where © is the density and k is a constant depending on the units used 
and the gas in question, Willis assumed that the temperature and 
pressure in free air varies according to tho relation that 

(E) = 0,670+0,5295(E 40, 26252)” 
where the relative temporature at a point is given as a function of the 
relative pressure, The relative pressure is known to be closely re- 
lated to the mass hoights of the atmosphere. The density cf tho atmos- 
phere at a2 point could then be computed after the pressure and temper- 
ature were obtained. (3,18) 

The derivations for this model are based upon the assumption 
that the atmosphere takes the form of concontric spherical surfaces, 
These may be described as horizontal strata which are of equal density 
and follow lovel surfaces, Any ellipticity of the actual physical sur- 
face is neglected, (3»11) 

Up to a zenith distance of 60 degrees a first approximation of 
the refraction formula may be given as 

R= tan Z 
where B is the barometric pressure at the point of observation and k is 
a constant whose value depends on the units of temperature and pressure. 
For a more accurate value of the refraction and for zonith distances 
over 60 degrees more terms must be added to this basic expression, 


The form used by the Coast and Geodetic Survey is 





Reo< (tan ah. 1-043) aee%2)| 
l4+c: 

which extends tho reliable application of the refraction formula to a. 
conith distance of 70 decroes or moro, Tho quantity ——= noarly oquals 
the rofractive index of the air, The quantity V- is based on the 
radius of curvature of the strata and propertios of the atmosphore at 
the station. This is roughly equal to 55 « This formula may be 
expressed as a part of an infinite series which is given as 


we es See 6 a 
R “yi (tan 201 C. sec-Z + C, sec'é Ce sec Z + y; 


4 
The Cy coefficient is tho quantity V~3cs and the remaining coofficionts 
depent on the constitution of the atmosphere, Even after these coceffie 
cients have beon determined for an assumed state of things, tho sories 
ceases to converge before the horizon is reached, Jt may be used 
practically for computations up to a senith distance of 83 or &4+ decreos, 
For the practical computation of the rofraction a logarithmic 


form of the above rofraction formula 4s used, This may bo represented 


as 
log, oR “ log | 9 (tan Zz) + log, Cs at 108, ntgaee'? 


14 ye Qel -1) (+8) 
where the quantities represented by tho logarithms aro terns of the 


27) 


refraction fornula given above, @ and ¥ represent departures fron 
the standard atmospheric conditions of pressure and temperaturo, The 
term containing Co is added as a correction to the °</(1+«5) term which 
varies slowly with modorate zonith distance, Howoevor, whon this term 
46 introduced it does not allow the whole rofraction equation to be 
taken as proportional to temperature and pressure, This is taken into 
account by the () -1)(¥+-%) torm in the expression, ‘This last term 
does not become important except at the largor zenith distances, (3e 11) 
The tables published for this rofraction model were derived for 





a pressure of 760 millimeters, a temperature of 10 degrees centigrade, 
a wave length for yellow light of 0.578 micron and a relative humidity 
of 60 per cent. Correction tables are provided for differences in the 
observed temperature and pressure from these selected values, These 
tables were used in the computations for this iiiestigations ee 

The computer program used for the Coast and Geodetic Survey Re- 
fraction Model was fairly simple and straightforward, The refraction 
values given in Table V of the U, S, Coast and Geodetic Survey Publi- 
cation No. 237(3) were read into the main program for every ten minutes 
of arc, Since these values were computed for a given set of conditions, 
correction coefficients for temperature and pressure variations had to 
be used, These were taken from Tables VI and VII of the above publi- 
cation, These correction coefficients were introduced into the sub- 
program using literal statements, These values were limited to a 
reasonable range of temperature and pressure for the reduction of the 
observed data, The range of these values was expanded for the compari- 
son of the three refraction models at various extremes of temperature 
and pressure, The subprogram was made up of three search loops which 
computed the correction values for the given temperature and pressure 
and the refraction value for the observed zenith distance, All of these 
factors were then multiplied together to obtain the astronomic refrac- 
Eon . 
3.3 The Baldini Refraction Model 

The rofraction equations in this model are derived for an 
object inside the atmosphere but the final form of the equations is 
such that they will hold for an object at infinity, Since it is assume 


ed that all stars are at infinity, this model may be used to compute 


10 








astronomic refraction. 

For his dorivations Baldini assumed a spherical earth with an 
atmosphere arranged in spherical layors., Ho adopted the conditions 
that the altitude of the atmosphere would always bo less that 64 
kilometers and the obServed zenith distance would bo less than 75 de- 
grees. This altitude limit represents the maximum height of strata over 
which the refracting power of the atmosphoro is assumed to be Zero, 

In this modol Baldini defines n as the indox of refraction of 
one layer in the atmosphere and n + dn as the index of rofraction of 
the next layer. Hoe states his basic refraction equation in the form 

Re = 1 ic a 

Sim {'t 
where Z is the zenith distance of the object in the first layor of the 
atmosphere.with an index of refraction n, To solve this equation for 
the refraction expressions for dn/n and tan Z aro required, To obtain 
these, since both are a function of the density of the atmosphore, 
Baldini used the relationship for density at various altitudes that 
in which ho is a constant amd fo is the density at the observer's loca- 
tion on the earth's surface, @ is the density at an altitude h in the 
atmosphere, After determining a value for h, this expression becomes 
eee ees 

where h is given in kilometers, This expression is similar in form to 
the relation proposed by Bessei, 1? 

Using these relationships the final form of the refraction 
equation for an object at infinity becomes 


R = Ao(no-1) tan Zo + Aj (no-1) tan°Zo + A»(no-1) tan-Zo 


with the quantities A,, A,, and A, given as 


11 








Ao = +0.99827 


A, = -0,00130 

Ap = +0.000006 
from tho derivations, Tho factor (No-1), withho as the index of refrac- 
tion at the observor's station, is computed from the equation of Barrel 
and.Sears, This computation is done in two steps with a preliminary com- 
putation being performed for the refractivo index of standard air at 
optical frequencies, This value, designated asg, is obtained using 
the formula 


(ng -1) 10? = 2.87604 + 16.288 , 0.1% 
Ve SS 


where }, represents the light wave lenzth in microns. This is then con- 





verted to the ambient conditions using the relationship that 


(No ~1) -~NWe-i P | 0,000000055e 
1+<§% 760 1+ at 





where © is the partial pressure of water vapor in tho air, - is the 
coefficient of expansion of air (0.00367) and Ut is tho temperature in 
degrees centigrade, The quantity C is obtained from the rolative humid- 
ity and a table of saturated vapor pressures for various temporaturos, (1) 
The computer program used for this refraction model is self- 
contained in its own subprogram, The computations using the Baldini Re- 
fraction Model were all done for an assumed relative humidity of 60 per 
cent and a light wave length of 0.578 micron. A table of saturated vapor 
pressures for selected temperatures was entered in the subprorram 
through the use of a literals statement. A search loop similar to those 
used in the Coast and Geodetic Survey Model program was employed to 
obtain the saturated value of © at the desired temperature, This 


value was modified for use in the program by multiplying it by the rela- 


ie 





tive humidity. The quantity (ng -1) was then computed and the astro- 
nomic refraction obtained from the final rofraction formula given above, 


3.31 Effects of Relative Humidity and Light Wave Longth 
on the Baldini Model 


An investigation was run on this subroutine to dotermine the 
effect of differences in the relative humidity and tho wavo length of 
light used for the observation, It was found that the rofraction varied 
very little for a range of relative humidity from 10 to 90 per cent. 
This was for a zenith distance range of 0 to 85 degrees with comparisons 
boing made at every five degrees of zenith distance, These results are 
included in Tablo 1 at the ond of this subsection. 

The offect of the wave length of light was examined for wave 
lengths ranging from 0,475 to 0.693 micron which is tho range of the 
visible spoctrum, There is a larger variation due to the wave length 
effect than was found with the relative humidity. The difterbhces wore 
as large as three seconds of arc at a zenith distanco of 75 degrees, 

The results of this investigation are included in Table 2 at the end of 
this subsection, 

From the results of theso investigations it has been assumed 
that the values used in the computation program were sufficiently accu- 
rate as there was vory little variation in the refraction due to the 
relative humidity and noarly all observable stars omit light in the 


yellow wave length region of 0.578 micron, 
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3.4 Tho Garfinkel Refraction Nodel 

The Garfinkel Model is the only one of the three used in this ine 
vestigation which was designed to give refraction values for zenith dis- 
tances up to and greater than 90 dogroos., Thore are some variations in 
the final form of the refraction equation for this model among tho 
agencies which use it. Two of thoso variations will bo given after a 
discussion of the original form derived by Garfinkol. 

Garfinkel selects a modol for the variation of the density of 
the atmosphere that is similar to the modol suggested by Ivory. Ina 
model of this type the temporature diminishes with a uniform rate re- 
gardless of the altitude, This type of modol is called a polytropic 
model, (5) 

With the perfect gas law assumed to hold for the earth's atmos- 
phere, tho density relationship is given in its final form as 

@ = [nile 
n+1 
where © is the relative density of the atmosphere at an altitude y 
above the obsorvation station. The quantity n is called the polytropic 
index. The relative temperature is considered to diminish at a constant 
rate to the point where tho altitude is Yae Above this point the rela- 


tive temperature is considored to be zero, This relation may be given 


by 
1 


{— eae = eS 


n+1i 


where T' is the constant rate at which the relative temperature de- 
creases, The relative temperature is then given as 


yes eS ae 
n+1 


where the condition at the observation station would be 
t= ee Yoo. 


16 





It may be seon that using this relationship for the temporaturo the 
value y, must satisfy the relation 
yma + 1 

for the relative temperature to equal zero at that me? 

From this polytropic rolationship Garfinkel develops a formula 
in which he expressos tho refraction as a power series which is a funce 
tion of the observed zenith distance, This equation is of the fom 

ReT,?u( Bo+By WB W435 +———) 

for zenith distances that are less than 90 degrees, The terms which 
make up this series will be explained in the following paragraphs, (9) 

In the derivation of the final form of this refraction equation 
there are some constants which appear, The most important of these are 
called Ky, @, and to These are affected by local conditions at the 
observing station as well as the choice of a value for the polytropic 
index, If a standard set of conditions is chosen and these values com- 
puted for them, these values can be converted to the conditions at the 
observation station by temperature and pressure pelationships. ‘9? 

Some factors must be defined before the above three constants 
can be computed, Whon the index of refraction is donoted by UW, at 
the observation station, a quantity <, is defined from 

Ko = (08-1) /2ny ° 
From this factor a value C, is defined as 
Cy = SAGlianc) ° 


Finally, a factor S, is defined by 


where Ho is the "height of the homegenous atmosphere for an ideal gas" 


and Lips 4s the radius of the earth to the observer's station, This 


- 
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hoif¢ht, Ho, is the height to which the atmosphore would extond abovo 
the oarth if its density wore the sano as at the point of observation 
and the prossure was equal to the observed pressure, (305) 

Using these values from the preceding paragraph, the constants 
for the standard sot of conditions can now bo dofined, Thoso values are 
given a double subscript to indicata that thoy are dorived for certain 
stipulated conditions and at standard temporature and pressure, Those 


constants are defined as 
Yoo «(2 cena (n + me 


Boo * 2°Go%002 (n + bee 


Koo * 2 CO, Soq n(n + yack 
using a@ chosen value for n, Tho variation of all tho factors dofinod 
for the standard conditions, excopt tomperature and prossure, is so 
small that it makos no significant changos in the values of these cone 
stants from station to station, Therofore, working valuos for these 
constants may be obtained at an obsorving station from 
} 
So “Woo sa 
By = Boo ‘ 
K = KooT o ow 
where Wis the "weather factor" and is obtained fron 
iar Yee 
where P, and To represent the ratios of the pressure and temporature at 
the observing station to those standard conditions for which the values 
were originally computed, Garfinkel computed those constants using a 
valuo of 5,000 for n and obtained 
Yoo = 8.1578 


Qoo = 0.03916 
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Koo = 4952" 
at a temporature of 0 degreos centigrade and a pressure of 760 milli- 
meters, (5) 
From these constants the B factors of the powor serios are de- 
fined as 
B= Koo¥ Gos al 
where 2S is a function of an angle theta, This angle is derived from 
the observed zenith distance by the relationship that 
cote= YT, * cotZ,. 
4 may also be expressed in torms of a power series of tan€/2(5) 
The refraction equation may be written for zenith distances 
greater than 90 degrees using the above quantities, From these factors 
two new values are defined, They are 


Z 
C, = 2 B- @ + 


2 cot 


and 


P 2 
y= © tan °©/2 
To“ 


The refraction for a zonith distance groater than 90 degrees is then 
obtained from the formula 

Re aK Cotly VaCV4C04V4-—-) 
which is also expressed in the form of a powor series, (5) 

Tho program which was used for most of the refraction computa- 
tions was that of the Garfinkel Modol as programmed by Abordeen, This 
program was translated from the FORTRAN II version into the SCATRAN Lan- 
guage, It basically follows the summations given by Garfinkel in his 
article to obtain the By factors, Tho refraction is then computed 
using Garfinkel's power series formula as given above, This program 


uses updated factors for Garfinkel's basic constants, These are 
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n= 4,256 
Koo= 4472.8 
B oo= 0, 044385 
Yoo= 8,7137 
which are computed for the same standard conditions of temperature and 
pressure as those of the Garfinkel article. In a comparison of the 
program using the two sots of constants, it was found that this newer 
set of valuos gives a smaller amount of refraction for oach selected 
zenith distance than when the original Garfinkel set was used, This 
difference has a magnitude of about 1.5 seconds at a zenith distance of 
75 degrees, This comparison is contained in Appendix I, The Aberdeen 
constants were used for all reductions and model comparisons in this 
project, 
Another form of the Garfinkel Rofraction Model is used by the 
Coast and Geodetic Survey. (4) This method computes the refraction from 


the formula 
R = Tow (Ny tan B/, + N tan? B/2 + Ny tan? B/2 
+ Ny tan a 2) 
where the constants are given the values 


Ny = 1050.61030 
No = 706.11502 
N4 = 262,06086 


Ny = 142.67293 
and the angle B is the same as the angle @ in the Garfinkel article 
formulas, The w& factor used in this mothod is tho same as that of the 
Aberdeen program, 
This refraction method was also programmed and used in some 


reductions, It gives slightly lower refraction values for zenith 
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distances up to 70 degrees and slightly higher values for the zenith 
distances over this value when compared to the Abordeon program, The 
difference amounted to about 0.02 seconds of arc at a zenith distance of 
75 degrees. The advantage of this program over that of Aberdeen is 

that it is much shorter and uses about ono-fourth of the computer time 
for the same purposes, Appendix I contains a comparison of the results 
of this subroutino with the Aberdeon values for one selected set of 
temperature and pressure conditions, 

Another vorsion of the Garfinkel refraction equation is that 
used by the Aeronautical Chart and Information Center (actc), (4) This 
obtains the refraction from the equation 

Re N, tand + No tan? + N3 tan? 
in which the angle 6 is the same as given in the Garfinkel article, 
This system uses a X value of 8.1578 which is the same as Garfinkel's 
original value. This form was programmed by using the values given in 
Garfinkel's tables for the various theta angles as the N factors in the 
refraction equation, This version was used only as a comparison for the 
other programs given above, It is included in the comparison given in 
Appendix I, 

This form of tho refraction equation was used as a basis for the 
adjustment program discussed in section 5, N factors were assigned for 
the purposes of the adjustment and a ¥Y factor of 8.7137 was used in that 
program, 

3.41 Comments on the Garfinkel Program 

The Garfinkel model was found to give the greatest difforence 
between the computed values of latitude, from the observed north amd 


south stars, of the three refraction models used, The Aberdeen sub- 
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routine was then used with different valuos of bee but holding all other 
factors constant, This would be equivalent to changing the coeffi- 
cients of the K,, and(Q,, terms which are computed from ¥/,,. These co- 
efficients are given as 2.0 by Garfinkel, It was found that a highor ¥ 
under these conditions gave less difference between the north and south 
star computed latitude valuos for all stations examined. There wore 
different values of the factor which gave the minimum spread for each 
station but they all were higher than the selected value, These values 
ranged from 8,78 to 8.88 for the minimum closure on each station, Howe 
over, the values of tho factor over the 8,78 level had tho effect of re- 
versing tho error in somo stations whon it was used to reduce all the 
observed stations, Therefore, any valuos higher than this one would 

be of questionable merit, This offect was to basically switch the 
positions of the north and south latitudes when compared to the com- 
bined value, This method of forcing less difference between the north 
and south stars is not given as a proof of any weaknoss in tho model 
but is considered worthy of note since there is an indication that an 
adjustment in the constant factors for the Ko, and@,, values by a 
Slight amount might result in a more accurato refraction model, This 
would be associated with a new higher value for the gamma constant that 
would be somewhere between the Aberdoon value and the 8,78 quantity 
montioned above, Much more study and rosearch would be roquired before 
it could be decided whether the noted changes were either required or 
practicable, The idea is put forth as a possible area for future re-~ 
pearch but should not be accepted as anything more meaningful than that, 


3.5 Comparison of the Refraction Values Obtained from the Threo 
Refraction Models, 


A comparison of the rofraction values for the Baldini Model, 


— 
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the Garfinkel Modol and the Coast and Goodotic Survey Model was made for 
various conditions and extremes of temperature and pressure, These com- 
parisons wore mado for a standard relativo humidity of 60 por cent, 
light wave length of 0.578 micron and over a range of zenith distances 
from 0 to 90 degrees, It should be noted that the form of the Baldini 
model goes to infinity at 90 degrees and so no comparison can be made 
there, 

It should also be noted in these comparisons that the Baldini 
Model was designed to be used only for observed zenith distances up to 
75 degrees and that the series used for the Coast and Geodetic Survey 
Model becomes quostionable after a zenith distance of 83 or 84 degrees 
is meacnedy ‘1s 3) The comparisons were made though to see how close 
the models did come, Table 3 lists the refraction values for various 
combinations of tomperature and pressure and also the difference in the 
obtained values between the models, This is dono using the Coast and 
Geodetic Survey Model as an arbitrary standard with which to compare 
the other two models, 

The comparisons show that the Baldini and Coast and Geodetic 
Survey Models tend to agree more closely throughout the range of tem- 
perature and pressure than the Garfinkel and the Coast and Geodetic 
Survey Models, However, at higher temperatures the refraction values 
given by the Baldini Model tend to become lower than those given by the 
Coast and Geodetic Survoy Model, At lower temperatures these values 
are highor than those given by the Coast and Geodetic Survey Model, 
The widest variation between the Baldini and Garfinkel Models occur at 
the higher temperatures, The relation betweon the Garfinkel Model and 


the Coast and Geodetic Survey Model at the higher zenith distances, 
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over 85 degrees, varies quito sharply with tho tenpoerature, In this 
area the Garfinkel Model gives highor values of refraction at the lowor 
tomperatures than the Coast and Geodetic Survey Model and lower ones at 
the higher temporatures, It was found generally that the refraction 
differonces appear to be more effected by the variation of temperature 
than those of pressure, All the modols give fairly close values for the 
refraction up to a zenith distance of about 50 degrees in which area the 
difference is about 0.3 seconds, 

For the comparisons in Table 3 the values of the refraction were 
rounded to the nearest 0,1 second, This is done since the tables for 
the Coast and Geodetic Survey Model are only given to this accuracy, 
Since the errors in this range are within the acceptable limits for 
second order work this factor does not tend to affect the results of 
this investigation, The Abordeen subroutine was used for the Garfinkel 


Refraction Model in this comparison. 
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TABLE 3 (CON'T) 


REFRACTION CGiPARISONS 


FOR THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast 
Zenith and Difference from Coast 
Distance Geodetic and Geodetic Survey Model 
derroes Surve Baldinti Garfinkel Baldini Garfinkel 


Temporature -5°C Pressure 760 mm Refraction in seconds 
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Temperature ae Pressure 787 mm Refraction in seconds 
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85 643.7 677.2 649.3 B55) 5.7 
90 2253.1 OO 2291.1 OO 38.0 
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TABLE 3 (CON'T) 


REFRACTION COMPARISONS 


FOR THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast 
Zeni th and Difference from Coast 
Distance Geodetic and Geodetic Survey Model 
decrees Surve Baldint Garfinkel Baldini Garfinkel 
Temporature 10°C Pressure 760 mm Rofrection in seconds 
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TABLE 3 (CON'T) 


REFRAC TION CG@iPARISONS 


FOR THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast 
Zenith and Difforence from Coast 
Distance Goodetic and Goodetic Survey Model 
derroos Surve Ba loasey Garfinkel Baldi ni Garfinkel 
Temporature 35°C Pressuro 760 mm Rofraction in soconds 
0 0.0 0.0 Of 0 0.0 0,0 
5 4.7 46 4.7 =On1 0.0 
10 9.4 9.4 9.4 0,0 0.0 
15 14.2 ey 14,3 0.0 0.1 
20 19.4 19.23 19.5 -0.1 Ova 
25 24.8 24.8 24.9 0.0 O74 
30 30.7 30.7 30.8 0.0 OF 
oD B7ee Oe 37.4 0,0 0.2 
LO dt 7 Hy, 5 ly .8 -0,2 O-d 
45 Dee pie pe! 53.4 -0,1 O72 
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85 540.7 566.3 536.2 ade A+. 5 
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Temperature 35°C Pressure 787 mm Refraction in seconds 
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Z5 2547 AY) Ages) 0,0 0.1 
30 Sl as: 91..8 31.9 0,0 0.1 
35 soul 3855 38.7 ~O.1 aol 
Wo “6.3 KO. 6.4 -0.2 ORS 
45 Doe 55.0 Dee ~0.1 0.3 
50 65.6 65.5 65.8 ~O.1 0.2 
55 78.5 78.4 78.8 ~0O.1 0.4 
60 OD ee 95.0 95.5 -0,.2 0.3 
65 11226 WAGs S 118.0 -0,3 0.4 
70 ples} 0 ad 149.8 150.6 ~0.4 0.4 
7) 202.6 201.9 203,1 -0.7 On 
80 302.1 301.0 302.2 ay Oar 
85 ESOee 586.5 555.4 2053 -—+.8 
90 1960.8 CYP 1818.7 Oe aah 1 
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Figure 2. Refraction Variation for the Coast and Geodetic Survey Model 
From 0 to 80 Degrees Zenith Distance 
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CHAPTER 4 


THE LATITUDE REDUCTION METHODS 

4.4 Background on the Reduction of Astronomic Data 
The latitude reductions for this projoct were done by two compue 

tation methods which were used as a check on each other for the final 
valuos obtained, The first method employed is that given by Thorson 
in the Second Order Astronomical Position Determination Manual of the 
United States Coast and Geodetic Survey, (9) This reduction method is 
based in turn on a version given by Chauvenet in A Manual of Sphorical 


and Practical Astronomy, 2? Two versions of this method were used with 
the only difference being the use of higher order terms in the second 


version to attempt to gage thoir effects on the final position, Tho 
second reduction method used is based strictly on tho use of spherical 
trigonometry and the astronomical triangle. A method outlined by 
Chauvenet was used in the final formulation of this mothod, (2) 

The observed data available for this investigation included 
observations for Polaris and a selected south star at each station. 

The maximum zenith distance difforences between the two stars at any one 
station was just less than two degrees, The latitude results were re- 
duced for both stars and then avoraged to obtain the combinod value 

for the station. A standard orror of the moan value obtained in avor- 
aging each of these separate latitudes was also computed, 

Astronomic latitude may be defined as the declination of the 
zenith of the observer, An alternate way of stating this fact is to say 
that the latitude is the altitude of tho obsorver's pole, Both of these 
definitions of latitude will be used in explaining the reduction 


mothods metioned Rbeve, (2? 
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The reduction methods used in this investigation are both based 
on the genoral spherical trifonometry formulas which are derived from 
the astronomical triangle. These goneral formulas may bo stated as 

sin h = sinfising + cos;cos§ cost 


cos t= sinh - sinssins 
cos ¢ cos & 


with the observor's latitude, h the altitudo, S the declination 
and t the hour angle of the star in question. The differences in the 
two methods aro in certain approximations that are made and in tho 
final method of use of the quantities, In the first reduction method, 
two separate means are used to reduce the north and south stars, while 
in the second reduction method,the samo genoral application is made to 
both ofoarag,, (oe 


4.2 Reduction Method Based on the Use of Polaris and A Circummeridian 
Star 


This method of reduction employs different methods of reduction 
for the pole star and the south star, Certain approximations are made 
to reduce the observed position of the star to the observer's meridian 
and thon employ the definitions given above to determine the latitude 
of the station. 

For the pole star, the latitude will be troated as the altitude 
of the pole. For this reduction,the declination will bo replaced by the 
polar distance which is defined as 90 degroes minus the declination of 
the star, Substituting this quantity into the first general equation 

given in Section 4,1 gives the expression 
sin h = sing cos P + cosq sin P cos t 
whore P represents the polar distance. As the value of P is small for 


Polaris (less than 2 degrees), the latitudo may be doveloped in a sories 
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form of ascending powers of P, This Serios oxpansion gives the option 
of using as many terms as are required for any desired precision, 
Chauvenst? obtains this expansion by writing tho latitude equation as 
2 = hoes 
where x is a small correction on the ordor of magnitude of PP, The 
series expansion for x is givon as 
x =P cos t + P@ sin? t tanh - 1/3 P? cos t sin@ t. 

This expansion is accurate to 0,01 seconds of arc, Aftor substitution, 
the final latitude expression becomes 

gd =h-Pcost +4 P* sin® t tanh = 1/3 P? cos t sin* t, 
A factor may be introduced into this equation to convort all values to 
the same units, This would be in the last two terms whero a value of 
Sine of one second of arc, to the appropriate power, is introduced as 
a conversion factor if P is in soconds, Thorson does not use the last 
term (Pp?) in his method of reduction, This limits the method to an 
accuracy for the final astronomic latitude of somewhere within one 
second of arc, (29) 

To reduco the circummeridian star, the latitude will be obtained 
from the declination of the observer's zenith, Sevoral observations 
of the star may be made close to the meridian and then reduced to the 
meridian. These altitudes observed near the meridian give nearly as 
accurate a result when reduced to tho meridian as if the observation 
were taken on the meridian, This accuracy is within one second of arc 
for the final latitude if the observations are within 30 minutes of 
time from the meridian and within 0.1 seconds of arc if the observations 
are within 15 minutes of time from tho moridian, (2) 


From the first general equation of Section 4.1, Chauvenet(2) 
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develops an expression for the zenith distance of the observed star, 
Using the trigonometric relation for the hour angle that 
cos t=i1-2 sin* ee 

the zenith distance may be obtained from 

cos Z = sin h + cos ycos (2 sin® pact) 
where 7 is the observed zenith distance corrected for refraction. / is 
introduced into the equation from the relation that on the meridian 

Z=ef- 5 
for a star south of the observer's zenith, Using the relationship that 
the altitude is the complement of the zenith distance, the expression 
may be written as 
cos Z = coS Z, - 2 cos sicos & sin“3t 

where 27.1 is the zenith distance reduced to the meridian, This equation 
is then developed into 2 power series which in its final form is 
expressed as 


Z = Am + &n 


—> 
i 


where 


A =z=cosycosS 
sin a4 


AS cot “, 


59) 
It 


nee See et 

Ne=esin ot 
in which m and n may be divided by the sine of one secon to put these 
terms in seconds of are. For practical computations, the value of 
may be used in place of 7% 4; to determine the valuos of A and B, An 
assumed value of the latitudo must also be used for the computations 


If a number of observations are taken, a series of theso equations are 


obtained and an average value of < 4 will result. The latitude will 
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then be obtained from the expression 


ve = Ss + oa 


which may be given the alternate form 
= Sot hy. 
Using the values determined above, this equation reduces to 
qf = § + 90° -h + Am - Bn, 

In practice a curvature correction for the time elapsed between 
the direct and reverse instrument sightings is introduced. This is given 
as 

my = 0,0001364 (aTS)* 
where AQT° is the olapsed time in seconds between the direct and reverse 
sightings. This correction is added to the m term of the above exe 
pression, Thorson does not use the last term of this oxpression (Bn) 
in his reduction, The value of the assumed latitude used in this method 
should be as close as possible to the expected final latitude value, 
(2,9) 


4.3 A Method for Obtaining the Astronomic Latitudes Using Basic 
Spherical Trigonometry 


This reduction method once again makes use of the first general 
equation given in section 4.1 for the solution to the astronomic 
triangle, The main difference between this method and the one given in 
the preceding section is that no series expansion is used here to obtain 
an expression for the latitude. Instead, two auxiliary variables, d and 
D, are introduced so that 

d sin D = sins 
dcos D= cos 8 cost, 
For convenience another auxiliary variablo, x, is brought into the 


reduction, This variable is defined by the relation 
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Gg = D=]0 
After substitution the general equation may bo expressed as 
d cos (f - D) = sin h; 
By eliminating d from the final solution, the final computation 
equations are obtained. These equations are given as 
tan D = tandsec t 
cos ¥ = sinh sin D cosecs 
pe Dow 

using the auxiliary variables defined above, The first equation deter- 
mines D which is taken to be always less than 90 degrees and positive 
or negative according to the sign of the tangent. The second equation 
is indeterminate as to the sign of \¥ since the cosine of +} and - ¥ 
are the same, Therefore, two values of latitude are obtained by the 
third equation; however, one of these values will be over 90 degrees 
and can be discarded, It was found that for Polaris, the D- vw value 
is to be used to obtain the latitude, For the south star, tho D +23 
value gives the desired result. In theso reductions a curvature 
correction was used for the south star to compensate for the elapsed 
time between the direct and reverse pointings of the instrument, This 
is of the form Am, where the quantities are the same as those intro- 
duced in the preceding section. This value was subtracted from that 
value for the south star latitude which resulted from tho D + % 
procedure, (219) 
4,4 The Station Reductions to Obtain the Astronomic Latitude 

Each station was reduced by three methods using each refraction 
model. The first reduction used the procedures given by Thorson, (9) 


The second reduction used this same method but added the higher order 





terms described in section 4,2 The third reduction omployed the method 
outlined in section 4.3 All of those reductions were done by making 
use of a computer program which used the subroutines for each rofrac- 
tion model, The Aberdoen subroutine was used here for the Garfinkel 
Model. These reduction results are given for each station in Table 4, 
As a check on tho accuracy of the reduction program, tho first sot of 
reduction results for the Coast and Geodetic Survey Model was compared 
to hand computed results and found to be in extremely close agreement. 
The slight differences, of a maximum value of 0.02 seconds of arc, may 
be attributed partially to the fact that the hand computations used 
averaged valuos of the observed altitudes to determine the trigonome- 
tric functions used in the reduction, In the program the function was 
computed directly for each observed altitude (corrected for the refrac- 
tion). The agreement botween the final values was regarded as being 
within the accuracy limits required for this investigation, 

In goneral, the final latitude values for the north and south 
stars, as well as the combined latitude, agroed closely for each of the 
three programmed mothods employed. The second and third reduction pro- 
grams were generally in close agreement in their results. It was found 
that these two programs reduced the differences between tho computed 
latitudes for the north and south stars by nearly 0.1 second when com- 
pared to the results of the first method, This change may be attrib- 
uted to the greater accuracy of these programs. 

Thorson states that if all field requirements are met, the final 
computations will show probable errors of less than ft 0.4 seconds of 
arc, 69) This would be equivelent to a standard error of about # 0.6 


seconds of arc, All of the observations used in this project wore with- 


ly} 





in this range, The standard orrors for the mean values of the latitudes 
determined by each reduction method are given in Table 4, 


4.5 The Effects of Refraction Model Errors Upon the Computed 
Latitude Values, 


The latitude may be computed from the observed zenith distance 
of a star using the definitions given in section 4.1 For the case of a 
star observed in the meridian, the latitude could be obtained from the 
equations 
D = NOs OO aaa) 

or 

2 is 2 
for a star observed north of the observer's meridian and 

eae tz. 
for a star south of the observer's meridian, In these equations ,»* is 
the latitude, S the declination and Z the true zenith distance, In the 
case of a circumpolar star observed at lower transit, the declination 
would have to be replaced by its supplement in the equations for the 
north star, The true zenith distance is obtained by adding the re- 
fraction to the observed zenith distance. This relationship is expressed 
by 

Z=4,+R 
where 2, is the observed zenith distance and R is the refraction correc- 
tion, Substituting this expression into the latitude equations, tho 
relationships are given as 

= (90° -5) + (90° - Z - R) 


or 


for the north star and 
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3° = e+ z. +R 
for the south star, 

It can be seen from the above expressions that the refraction 
term is negative in them to obtain the latitude from the north star and 
positive when latitude is obtained from the south star. Since the signs 
of the refraction terms are opposite for the north and south star lati- 
tude reductions, any constant error in the refraction model from which 
this term was obtained would cause an opposite effect on the latitudes 
computed from the two stars. If the refraction model gives a value 
for the refraction correction which is lower than it should be for all 
zenith distances, the latitude value computed for the north star will 
be greater than it actually is, and the latitude value from the south 
star will be less than it actually is. The converse case is true if 
the refraction model gives refraction values that are consistently high- 
er than they should be, 

In the observations which were used for this investigation, the 
recommended procedure had been followed in the selection of the stars, 
(9) This procedure calls for selecting the north and south stars at 
nearly equal zenith distances, When this is done, since we have seen 
that the refraction gives an opposite effect for the two stars, the 
average value of the two computed latitudes should minimize any errors 
due the refraction model, Since Polaris was used as the north star in 
these observations, south stars with a zenith distance near 50 degrees 
were selected, 

Even when this averaging of the two latitude values is done to 


minimize the refraction errors, it must be pointed out that there will 


still be errors present in this final latitude value due to observation 
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and instrument conditions, but these should be smaller and in a random 
distribution, It should bo noted that somo refraction model crror will 
still be prosent after the latitudes are averaged due to the fact that 
the stars are not at exactly the same zenith distancos and possibly due 
to atmospheric conditions at the time of the observation. 

Generally, then, it will be stated that the best refraction 
model would give the minimum difference between the values of latitude 
computed from observations of the north and south stars at any given 
time. 


4.6 REvaluation of the Results Obtained for Latitude from the 
Observation Data, 


It was found that the standard error of the mean value of the 
computed latitude was consistent for the three refraction models ine 
vestigated, This is true for the latitude values computed for the north 
and south stars as well as the combined latitude. No significant 
change occurs in any of theso standard error values due to the refrace 
tion model used and, in fact, very little change occurs when no refrac- 
tion model at all is used. These values can be found in Table 4, 

The combined latitudes for each refraction model were found to 
be in very close agreement for each reduction method employed, with no 
Significant difference noted for second order results, The widest 
variation among these valuos is in the range of 0,02 seconds, Some 
variation was found in the computed values for the latitudes that wero 
reduced from each of the two stars. In this light, it was found that 
the Coast and Geodetic Survey Model gave the least difference between 
the north and south star values of latitude, The Baldini Model gener- 


ally gave just a slight bit more spread with the Garfinkel Model 


giving the widest range between these two values. This difference in 
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spread was found to closely equal the refraction differences, In all 
cases, the difference between the combined latitude value and those of 
the north and south stars was too great to justify using the observation 
or just one star to determine the astronomic latitude of a station. 

This is using the criteria stated in Section 4.4 for the accuracy of 

the observations, 

The reductions were done for both the Aberdeen set of constants 
and the original ones suggested by Garfinkel when the Garfinkel Model 
was used, It was found that the Aberdeen values gave about one second 
less in the differonce between the north star and south star latitude 
values for each station. This was true for all of the stations reduced, 
The Coast and Geodetic Survey Garfinkel program was used for the reduc- 
tions also, and it was found that slightly less difference was obtained 
between the north and south star latitudes than when the Aberdeen pro- 
gram and values were used. This diffoerenco was not significant for 
the purposes of this investigation, having 2 magnitude of about 0,03 
seconds of arc. 

It sould be noted that the accuracy of the combined latitude 
would be affected to 2 groater degree by the refraction model as the 
difference in the zenith distances of the observod stars increased, 

At large zenith distances, the refraction modol would also have more 
effect on the standard crrors of the mean values obtained since tho re- 
fraction becomes much grcater as the zenith distance incroasas, 

From the results given in Table 4, it can bo stated that for 
second order latitudes observed using a north star and a south star 
with nearly equal zenith Aeecanceen any of the three refraction models 


investigated would give accoptablo results. 
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TABLE 4 


LATITUDE REDUCTION RESULTS 
USING THE 
THREE REFRACTION MODELS 


Coast and 

Refraction Geodetic 

Model None Survey Baldini Garfinkel 
Sita C..0N > Carr z 
Temperature: 20.88 C 
Pressure: 735-2 mm 
Latitude 59 oe 39° 32! 39° 32! 
North Starr : i tf tt 
Program 1 18.612 12.088 12.051 11.824 
Std Error(+) Ol, sieul 0.384 0.385 0.386 
Program 2 121 12S Se 11.927 
Std Error(+) 0,385 0. 386 0.386 
Program 3 12,188 12.150 11.923 
Std Error(+) ; 0.385 O23 0. 386 
South Star 39 BAC 
Program 1 08.132 14,092 14,097 14,322 
Std Error(+) 0.492 0.493 0.493 0.493 
Program 2 14,092 14,098 14,323 
Std Error(+) 0,493 0.493 0.493 
Program 3 14,098 14.105 14, 330 
Std Error(+) 0.493 0,492 0,492 
Combined 39° 32! 
Program 1 la. oCe 13.070 13.072 138072 
Std Error(+) 02257 0.257 0.257 0.257 
Program 2 13,141 1352125 he Less 
Std Zrror(+) 0.257 0.257 0.257 
Program 3 13.142 1 aaa27 1327 
Std Error(+) 0.257 0.257 0.257 
Difference 
Nand S 
Latitudes " Tt | tf 
Program 1 130.480 2.003 2.046 2.498 
Program 2 1.900 1.945 2.396 
Program 3 1.910 1.955 2.406 





USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Geodetic 
Model None Survey Baldini Garfinkel 
Station: Foust 
Temperature: 16.90°C 
Pressure: 73000 000 
Latitude 90° ae 39° «4a! 39° at 39° 4a! 
North Star 7 tf " tt 
Program 1 16.067 09 . 328 09.275 09.079 
Std Error( +) 0.726 0.719 0.719 0.719 
Program 2 09.428 09.376 09.184 
Std Error(+) 0.720 0.719 0.719 
Program 3 09.423 09.375 09.177 
Std Error(+) 0.720 - 0.720 0.720 
South Star 39° =o! 
Program 1 07.803 11.469 11555 11.736 
Std Error(+) 0. 583 0.584 0.585 0. 584 
Program 2 11.472 11.556 11,742 
Std Error(+) 0, 584 07585 0,584 
Program 3 11.474 Lie 11.774 
Std Error(+) : 0. 584 0, 584 0,584 
Combined 39°—s—«& 
Program 1 11.934 10. 396 10.415 10.408 
Std Error(+) 0.424 0, 447 0.416 0,416 
Program 2 10.449 10.468 10.460 
Std Error(+) 0.417 0.416 0.416 
Program 3 10,448 10,446 10.461 
Std Error(+) 0.416 0.416 0.416 
Difference 
Nand S 
Latitudes BY | y "7! 
Program 1 128,264 21a 2.278 2.659 
Program 2 2.044 2.180 200be 
Program 3 2.052 2.184 2.566 


TABLE 4 (CON'T) 


LATITUDE REDUCTION RESULTS 
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TABLE 4 (CON'T) 


LATITUDE REDUCATION RESULTS 


USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Geodetic 
Model None Survey Baldint Garfinkel 
Station: Haines 
Temperature: 15,88°C 
Pressure: 740.1 mm 
Latitude 39° 33" = 39° 328 39° 32! 39° 32" 
North Star tt tf +f tt 
Program 1 54.795 47,310 47 253 47.032 
Std Error(+) 0.374 0.377 Os O07 05377 
Program 2 47,396 47,337 47.116 
Std Error(+) O2o77 0.378 0.378 
Program 3 47, 390 47,334 47.112 
Std Error(+) 0.378 0.378 0.378 
South Star 39° 33! 
Program 1 45,886 50.786 50.841 51.054 
Std Error(+) 0,220 0.220 0.220 0.220 
Program 2 50.793 50.864 51 059 
Std Error(+) 0.220 07220 C220 
Program 3 50.78 50.884 51.057 
Std Error(+) 0.222 0.221 0.221 
Combined 39° 32! 
Program 1 50.340 49.050 49 .047 49.042 
Std Error(+) 0.162 Og162 0.162 O62 
Program 2 49,092 49.092 49 .086 
Std Error(+) 02101 Gelo2 Oni61 
Program 3 49.090 49.088 49 .083 
Std Error(+) Oriel 0.161 0.161 
Difference 
Nand S 
latitudes i 1% tt tt 
Program 1 128.909 3.476 3.588 4.022 
Program Z Cae 7 34508 3.943 
Program 3 3.398 3.610 3.945 





TABLE 4 (CON'T) 


LATITUDE REDUCTION RESULTS 


USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Goodetic 
Model None Survey Baldini Garfinkel 
Station: Head " 
Temperature: 15.88 C 
Pressure: 738.9 mm 
Latitude 39° 32! 39° 30! 39° 30! 39° 30! 
North Star ? t! tf " 
Program 1 01.838 532362 BS relent Seo 
Std Error(+) 0.304 0.294 0.294 0.295 
Program 2 53.464 Bin 5185) 53.193 
Std Error(+) 0,294 0.294 0.294 
Program 3 53.459 53.380 53.188 
Std Error(+) . 0.294 0.293 0.294 
South Star 39° 29! 
Program 1 50.054 55.740 55.809 55.993 
Std Error(+) 0.347 0.347 0. 346 0.347 
Program 2 55.747 Sous 55.998 
Std Error(+) 0.347 0,346 0.347 
Program 3 65a745 Sono 55.996 
Std Error(+) 0, 346 0. 346 0. 346 
Combined 39° 30! 
Program 1 55.944 SH. 561 54.554 54.551 
Std Error(+) 0.275 0.266 0.265 0.265 
Program 2 SH.604 54.599 54.595 
Std Error(+) 0.265 0.264 0.264 
Program 3 54.604 54.595 54.592 
Std Hrror(+) 0.265 0.265 0.265 
Difference 
N and S 
Latitudes '? f tf! tf 
Program 1 1312755 Zep S518: 2.508 2.882 
Program 2 2.283 2.429 2.804 
Program 3 2.287 2.432 2.808 





TABLE 4 (CON'T) 


LATITUDE REDUCTION RESULTS 


USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Geodetic 
Model None Survey Baldinit Garfinkel 
Station: Hoyt . 
Temperature: 19.21 C 
Pressure: 738.6 mm 
Latitude 39° 4ot 39° (39! 39° 39! 39... 391 
North Star " Z a” se 
Program 1 26.719 19.949 19.953 19.740 
Std Error(+) 0.420 0.456 0.455 0.455 
Program 2 20.047 207052 19.839 
Std Error(+) 0.452 0.454 0.454 
Program 3 20.045 20.049 19.838 
Std Error(+) ; 0.453 0.454 0.455 
South Star a 235" 
Program 1 18.996 22. 588 22.618 22.818 
Std Error(+) 0,382 0.378 0.378 0.378 
Program 2 22.589 22.622 22.818 
Std Error(+) 0,378 05378 0,378 
Program 3 22, 589 22,622 22,824 
Std Error(+) , 07376 0.379 0.378 
Combined 39 39! 
Program 1 22,057 21.266 21.283 21.279 
Std Error(+) 0.339 0.347 0. 348 0.348 
Program 2 21,319 21.336 Vanes BS)5) 
Std Error(+) 0,347 0.348 0. 
Program 3 214517 21.334 21.329 
Std Error(+) 0.347 oe on 
Difference 
N and S 
Latitudes " W " 1" 
Program 1 1277 2e 2.639 2.666 3.078 
Program 2 2.542 2.570 2.994 
Program 3 2. 544 2.573 2.987 
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USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Geodetic 
Model None Survey Baldini Garfinkel 
Station: Imler 
Temperature: 20.53 °C 
Pressure: 735.7 mm 
Latitude 39° (332! 39°31! 39° 31! 39° 34° 
North Star ti! 1! tt tt 
Program 1 43,329 Bon 78'S BO e728 36.498 
Std Error(+) 0.295 0,302 0. 302 0.302 
Program 2 36.834 36.820 36. 597 
Std Error(+) 0. 302 0.302 0. 302 
Program 3 36.829 Bere 36. 594 
Std Error(+) 0. 302 0. 302 0.302 
South Star 39° = 30! 
Program 1 36.789 39. S41 39.576 39.785 
Std Error(+) 0.187 0.187 0,187 0,187 
Program 2 39. SH5 39.577 39.787 
Std Error(+) 0.187 0.187 OF187 
Program 3 39.552 39. 584 39.794 
Std Error(+) 0.487 0,187 0.187 
Combined 39° 31! 
Program 14 40.058 Baie 1 37, 38.149 38.142 
Std Error(+) 0.187 0.189 0.189 0.189 
Program 2 38.190 38.199 38.192 
Std Error(+) 0.189 0.189 0.189 
Program 3 38.190 38.201 38.194 
Std Error(+) 0.189 0.189 0.190 
Difference 
N and S 
latitudes tt tt tt 1 
Program 1 126, 540 2.805 2.854 3.288 
Program 2 24710 2.756 3.190 
Program 3 2. fee 2.768 3.200 


LATITUDE REDUCTION RESULTS 


TABLE 4 (CON'T) 
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LATITUDE REDUCTION RESULTS 


USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Geodetic 
Model Nono Survey Bald4ini Garfinkol 
Station: Lanun , 
Temperature: 12.98 C 
Pressure: 733.6 mm 
latitude 39” 4o' 397 391 39° 39! 39° 39! 
North Star if 9 ft 1 
Program 1 50.997 43,111 42.998 42,825 
Std Error(+) On271 0.25% 05255 0.255 
Program 2 43,213 43,101 42.928 
Std Error(+) OnZ 0.255 0.255 
Program 3 43,208 43,096 42.923 
Std Error(+) F 0.255 0.255 0.255 
South Star 39 
Program 1 35.678 190575 43,686 43,862 
Std Error( +) 0.199 0.199 0.200 0.199 
Program 2 43,578 43.690 43,862 
Std Error(+) 0.199 0.199 0.199 
Program 3 43,580 43.693 43,876 
Std Error(+) 0,200 0.199 0.199 
Combined 39° 639! 
Program 1 43,388 43,343 43,341 43, 343 
Std Error( +) 0.139 0, 134 0.130 0.130 
Program 2 43,395 43,395 43,393 
Std Error(+) 0.130 0.130 0.130 
Program 3 43,393 43.393 43,395 
Std Error(+) 0.130 0.130 0,130 
Difference 
N and S 
Latitudes if t t 1 
Program 1 135.219 0,464 0,688 1.036 
Program 2 0. 366 03570 0.934 
Program 3 e372 0.597 0.944 





TABLE 4 (CON'T) 


LATITUDE REDUCTION RESULTS 
USING THE 
THREE REFRACTION MODELS 


(Continued ) 
Coast and 
Refraction Geodetic 
Model None Survey Baldini Garfinkel 
Station: Meinfeller 
Temperature: 16,.85°C 
Pressure: - 737.3 ™m 
Latitude 39° 39! 39° «37! 39% 37: 39° 371 
North Star 1 8] ih " i 
Program 1 07.178 59.370 59.318 59.119 
Std Error(+) 0.239 0.236 0.236 0.236 
Program 2 59.452 9.399 59.201 
Std Error(+) 0.236 0.236 0.236 
Program 3 59 449 59.394 59.196 
Std Error(+) 0.236 0.236 0.236 
south Star 39° «36! 39 38! 39 38! 39 38' 
Program J 55.221 00.813 00.877 01.069 
Std Error(+) 0,248 0.245 0.245 0.245 
Program 2 00,818 00.884 01.074 
Std Error(+) 0.245 0.245 0.245 
Program 3 00,818 00,884 01.074 
Std Error(+) 0,245 0.245 0.245 
Combined 39° 838! 
Program 4 01.199 00.092 00.097 00.094 
Std Error(+) 0.159 0.159 0.159 
Program 2 00.137 00.140 003137 
Std Error(+) Oni 0.159 0.159 
Program 3 00.133 00.140 00235 
Std Error(+) 0.159 0.159 0.159 
Difference 
N and S 
Latitudes " is i: 4 
Program 1 131.956 1 4d 1.2555 1.950 
Program 2 14.806 1.485 1.873 
Program 3 1.070 1.490 1.878 


DD 
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LATITUDE REDUCTION RESULTS 


USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Geodetic 
Model None ourvey Baldini Garfinkel 
Station: Owens 
Temperature: 21.69°C 
Pressure: 736.4 mm 
Latitude 39° hot 39° 39! 39° 39! 39° 39° 
North Star e t u 
Program 4 42.374 26)... en) 36.216 35.982 
Std Error(+) 0, 348 0, 360 0.361 0.361 
Program 2 36.254 86 7815 36 084 
Std Error(+) 0.360 0,361 0.361 
Program 3 pene se 36.315 36.082 
Std Error(+) 0.360 0,361 0,361 
South Star 39°38 
Program 1 33.001 39.283 39.222 39.455 
Std Zrror(+) 0,526 0.514 oroal’s 0,514 
Program 2 39.295 39.235 39 469 
Std Error(+) 0.509 0.509 0.509 
Program 3 39.275 39.218 39 448 
Std Error(+) 0.519 0.519 0.519 
Combined 39° 391 
Program 1 37.685 37.716 37.718 BYSVAIE 
Std Error(+) 0.301 0.282 0.281 0.281 
Program 2 37.774 BYES: 37.774 
Std Error(+) 0.282 Oreo 0,281 
Program 3 37782 37.766 37.766 
Std Error(+) 0,285 0,284 0.284 
Difference 
Nand S 
Latitudes "7 ! ? f 
Program i 129-573 3.433 3,006 3.473 
Program 2 3.042 2.920 3.385 
Program 3 B5025 2.903 Boils 





USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Geodetic 
Model None ourvey Baldini Garfinkel 
Station: Persinger 
Temperature: 19.72°C 
Pressure: 741.2 mm 
Latitude p9S3! 39° Sor 39° 32! 39° 32! 
North Star "1 "1 tt iW 
Program 1 20.578 13.098 13.039 12.820 
Std Error( +) 0,264 OF275 0.276 0.276 
Program 2 13.199 Tet WZ2e921 
Std Error(+) 0.276 0.277 O2277 
Program 3 133197 135355 12.917 
Std Error(+) " 68275 0.276 Oro 77, 
South Star 39°—~—ié«“‘«S23' 
Program 1 09.767 14.775 14, 838 15.046 
Std Error(+) 0.567 0.561 0.561 Om5o4 
Program 2 14,782 14,785 15.055 
Std Error(+) 0.559 0.559 0.559 
Program 3 14.776 14,842 15205) 
Std Error(+) 2 0.562 0,562 0,562 
Combined 39—ss 32' 
Program ij Ey 11 fal Uo! oi BS) Si Nosh oS 
Std Error( +) 0, 314 0, 307 0. 307 On007 
Program 2 13.989 13.992 13.985 
Std Error(+) On307 0.306 0,307 
Program 3 13.987 13.989 13.985 
Std Error(+) 0.307 0.307 0.307 
Difference 
Nand S 
Latitudes tt tt tt 1 
Program 1 1e0ee14 Led 1.779 2.227 
Program 2 1.503 1.708 YZ NSS 
Program 3 1.580 1.706 2.134 


TABLE 4 (CON'T) 


LATITUDE REDUCTION RESULTS 
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USING TRE 
THREE REFRACTION MODELS 
(Continued ) 
Coast and 
Refraction Geodetic 
Model None Survey Baldini Garfinkel 
Station: Tarbill 
Temperature: 24,01°C 
Pressure: 737.9 mm 
Latitude 39° 333! Sele 6 39° 32! 39° 32! 
North Star u " " " 
Program 1 08,683 02.767 02.788 02.532 
Std Error(+) 0.395 0.385 0.385 0.385 
Program 2 02.870 02.887 02.633 
Std Error(+) 0.385 0.385 0.385 
Program 3 02.685 02.884 02.630 
Std Error(+) 0.385 0,385 0.385 
South Star 39° 331" 
Program 1 02.341 04, 563 04.570 04,806 
Std Error(+) 0.505 0.510 0.510 0.510 
Program 2 04.570 04,573 04,813 
Std Error(+) OsSad: Cent OaA 
Program 3 04.571 04,578 04,817 
Std Error(+) 0, 509 0,509 0.509 
Combined 39° =632! 
Program 1 05.510 03.665 03.677 03.670 
Std Error(+) 0.165 0.154 0.154 0.154 
Program 2 03.720 03.730 03.723 
Std Error(+) OR ol os. 0.154 0.154 
Program 3 03.720 03.730 03.722 
Std Error(+) 0.155 0.154 0.154 
Difference 
N and S 
latitudes tt tf? wT W 
Program 1 126.343 1.796 neve 2.274 
Program 2 arAele 1.686 2.180 
Program 3 1.706 1.694 Pap MeiGs 


TABLE 4 (CON'T) 


LATITUDE REDUCTION RESULTS 
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USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast ard 
Refraction Geodetic 
Model None Survey Baldi ni Garfinkel 
Station: Taylor 
Temperature: 21.45°C 
Pressure: 732.0 mm 
Latitude 39° 44! 39° 4o! 39° oO! 39° 4oF 
North Star tf? " 1A wt 
Program 14 38 ,837 32.646 32.639 32.407 
Std Error(+) 0.680 0,689 0.690 0.690 
Program 2 32.730 32.722 32.490 
Std Error(+) 0.690 0.691 0.691 
Program 3 S27 26 322716 32.490 
Std Error(+) 0.690 0.691 0.691 
South Star 39° 39! 
Program 1 313223 DD gag 35.355 B55570 
Std Error(+) 0,294 0.291 0.291 0.291 
Program 2 15 SOLS. Boa so2 352585 
Std Error(+) 0.291 0.291 0.291 
Program 3 35.334 35. 360 Be aes 
Sta Error(+) 0,292 0.292 0.292 
Combined 39° hor 
Program 1 34.977 33.987 33.995 33.992 
Std Error(+) 0,289 0.296 0,296 0.296 
Program 2 34.031 34.038 34.036 
Std Error(+) 0.296 0.297 0.297 
Program 3 34.031 34.038 34.035 
Std Error(+) 0.295 0.296 0.296 
Difference 
N and S$ 
Latitudes "? Ai tf ? 
Program 1 127.715 2.683 2.716 3.168 
Program 2 25507 2.641 3.095 
Program 3 2.606 2.642 3.093 


TABLE 4 (CON'T) 
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LATITUDE REDUCTION RESULTS 


USING THE 
THREE REFRACTION MODELS 
(Continued ) 
Coast ard 
Refraction Geodetic 
Model None Survey Baldini Garfinkel 
Station: Williams 
Temperature: 16.75% 
Pressure: 730.4 mm 
Latitude 39° 33" = 39"—-332! 39a ee 39° gan 
North Star tf tt W t! 
Program 1 BV CON0. 25.497 FES PG GAT: 25.179 
Std Error(+) 0.367 0.370 0.370 0.370 
Program 2 25.524 25.404 254207, 
Std Error(+) 0.370 0.370 0.370 
Program 3 25.519 25.401 25ces 
Std Error(+) 0, 370 0.371 On 37 
South Star 39° 31" 
Program 1 26.538 28.509 28.619 28.800 
Std Error(+) 0.607 0.599 0.599 0.599 
Program 2 282510 28.626 28.806 
Std Error(+) 0.597 0.597 0.597 
Program 3 28.515 28.626 28.805 
Std Error(+) 0.601 0.601 0.601 
Combi ned 39° «32° 
Program 1 29 .874 27.002 26.999 26 .987 
Std Error(+) 0.343 0.336 0.336 05355 
Program 2 27.019 27.016 27.006 
Std Error(+) 0.334 0.334 0.334 
Program 3 2720 1C 2720s 27.004 
Std Error(+) 0.335 0.336 0.330 
Difference 
Nand S 
Latitudes ff 7 Ui Hh 
Program 1 126.672 3.012 3.242 3.621 
Program 2 2.988 Bees 3.600 
Program 3 2.996 Bnece 3.602 
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CHAPTER 


A SUGGESTION FOR AN ADJUSTMENT PROCEDURE 
USING A FORM OF THE GARFINKEL MODEL 
5.1 Background for the Adjustment Method 

Since the atmosphere and its refraction effects can be varied 
by the conditions which provail concerning weather, season or turbu- 
lance at any given time, there will be some factors which can not be 
accounted for by the assumptions for which the refraction model has 
been constructed, These effects will not be exceptionally large, but 
their presence can effect the results obtained for the refraction at a 
station. There also may be some errors in the refraction model itself 
that can be minimized by the use of an adjustment procedure. It might 
then be possible to construct an adjustment program which would perform 
an adjustment on certain of the refraction coefficients in order to give 
refraction results which will reflect the conditions at the station 
where the observations were taken. If many stations were treated in 
this manner, then the coefficients iced could be examined for any general 
trends exhibited. This could lead to revisions and a more accurate re- 
fraction model, 

To truly reflect the differences in the refraction coefficients 
and to take into account the effect of any observation errors, the 
General Mothod of Adgustmentss!°? should be used for this typo of pro- 
ject. This method considers both the variation of assumed parametors 
and the effect of the observations, This system was used in an in- 
vestigation conducted to determine the feasibility of this suggested 
type of station adjustment. 


In the investigation conducted, two variations of the samo 
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basic form were employed. Tho first form used the condition that the 
values of latitude computed from the observation of the north and south 
stars should be the samo, The socond form iS more general and assumes 
that the first general equation derived from spherical trigonometry and 
the astronomical triangle, given in Section 4,1, should be satisfied 
for the observed star after correction for the refraction. These two 
methods will be discussed individually in the sections to follow. 
5.2 The Form of the Garfinkel Refraction Model Selected 

The form of the Garfinkel Refraction model suggested by ACIC 
(4) was chosen for this adjustment procedure, This form was discussed 


in Section 3.4, The refraction is thon obtained from the relation that 


RaW VTo (N, tano +N, tan 3o +4 N, ta-6) 
where 
ao Pe 
W= P/ no 
tane = To Tan Z 
ae Oo 


and the value of § is chosen as 8,7137. All of these quantities have 
been previously defined, For the purposes of an adjustment, it was 
considered that the pressure, temperature and gamma factor were con- 
stant. The gamma factor used hore is not the one suggested in the ACIC 
article but is that of the Aberdeen and Coast and Geodetic Survey 
Garfinkel programs which was found to give less difference in the com- 
puted latitudes of the north and south stars, Since tan Z, is the same 
as cot ho, the observed altitude was substituted into the adjustment 
equations for the observed zenith distance, The assumed parameters 
become Ny, No and No from this oquation. These factors were assumed 


from the refraction results given by the Garfinkel Model for zenith 
distances in tho 40 to 60 degree range, with an approximate mean 
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value taken for each factor, These assumed valuos are given in 
Appendix II along with an explanation of the adjustmont procedures, 


5.3 Adjustment Method Using a Mathematical Structure Dealing with 
the Latitudes Computed from the North and South Stars 


In this method the mathematical structure is 
ar _V'c =-() 
where the subscripts denote from which observed star the latitude 
was computed, The declination is considered as a fixed quantity in 
this adjustment, The observed quantities are considered to be the hour 
angle and altitude of both the north and south stars, This results in 
a2 B matrix with dimensions of 4 by 8 which consists of the partial 
Piri vatives of the mathematical structure with respect to each of the 
observed quantities. The weight of all of the observations has been 
considered to be equal, The parameters are considered to be the three 
factors from the refraction equation and the assumed latitude used for 
south star computations, This gives an A matrix of dimensions 4 by 8 
which consists of the partial derivatives of tho mathematical structure 
with respect to the parameters. No weights were assigned the paramoters 
for this adjustment. The basic formulas used to obtain the corrections 
to the parameters are given as 
AX + BV+W= 0 
X= -(a'utt A) “t Aw 
Wea eles 
where the X matrix gives the corrections to the assumed papaniiwns, 1°? 
The W matrix is computed for each observation and each element is the 
number which would have to be added to the equation, computed from the 


observed values and assumed parameters, to fulfill the condition given 


by the mathematical structure. The compononts for these matricos and 
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and some additional information on this adjustment method are given in 
Appendix II. 
5.4 Adjustnent Method with a More General Mathematical Structure. 

The second method used the same computed quantities but a 
different mathematical structure that pives a condition which each ob- 
served star must fulfill. This mothod has the same parameters and fixed 
values as the previous method but has a B matrix with the dimensions 2 
by 16, since there are only two observed quantities for each observation, 
These are the hour angle and the altitude of each observed star. The 
same final adjustment matrix equations are used for this form as for 
the previous one, This is a more general form and if it were found 
workable would give morse room for variation in the stars obsorved as 
two south stars could be substituted for the north and south star of 
the previous method, The same thing could be done using two north 
stars also. This method could be expanded to use more data than that 
provided by just two stars because of its general form. 

The investigation for this model used only two stars due to the 
lack of any other available data, The matrix components used and other 
information on the adjustment are given in Appendix II. 

5.5 Results of the Adjustmont Method Investigation 

The results obtained from the investigation of these two suggest- 
ed adjustment methods are not in any way conclusive in a positive senso, 
It was found that the observed data were too strongly correlated to 
give any useable results, This was caused by a vory unstable MN matrix 
which in turn caused the inverse M matrix to be non-symmetrical and 


therefore caused the adjustment results to be unuseable, This corre- 


lation may be due to the fact that all of tho observed altitudes 
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were within two dogrees of cach other. 

Thereforo, nothing can be said in the way of constructivo con- 
clusions from the adjustment results except that they do not work for 
the data available for this projoct. A suggested positive test for 
these methods would bo to obsorve stars, both north and south of the 
zenith, at difforent zenith distances and hour angles and then uso the 
adjustment procedures, The reduction procedures for theso observations 
would not be the same as those used in this project and new equations 
would have to be written with the mathematical structure given in 
section 5.3, as the forms given in Appendix II are for the case of Polaris 
and 2 circummeridian star, A difference in zenith distancos of from 
20 to 30 degrees is recommended, These zenith distances should be ob- 
served up to about 70 degrees for the study, 

If this type of station adjustment for the refraction coecffi- 
cionts was found feasible, then a Seateee of obsorvations, say on two 
stars, could be taken beforo and aftor the latitude observations at the 
station, It would be bost if all observations wore taken at a singlo 
station over a long period of time to determine any consistent diffoer- 
ences in the refraction model, However, data taken at different 
stations could be used in the project for comparison purposes, 

1f it were proved to be feasiblo, this type of station adjust- 
mont might oventually load to a situation where the latitude of a 
station could bo detormined, for second order accuracy, with the obsor- 
vation of only one star. The station adjustment could also be employed 
as a tool for determining seasonable variations and thoir effects on 


the refraction model, Therefore, the idea does have some merit and 


should be investigated rather throughly before being completely 
discarded, 





CHAPTER 6 


SUMMARY AND CONCLUSIONS 

The basic objective of this project was to determine if the 
refraction model used in second order astronomic latitude determinations 
had any effect on the final latitude results. The method of observing 
both a north and south star at nearly the same Zenith distance was used. 
It can be stated, based upon the results of this project, that the final 
combined astronomic latitude is not significantly affected by the re- 
fraction model used, This is in reference to the three refraction 
models used in this investigation. It may also be noted that the stand- 
ard error of the mean for the computed latitudes is not affected to any 
Significant degree by the refraction model used in reductions. 

It was found that with the three refraction models used, that it 
would not be feasible to observe just one star, north or south, with 
an expectation of obtaining results of acceptable second order accuracy. 
This is based on the consideration that a standard error of = 0.6. 
seconds of arc would be the acceptable criteria, The spread given be- 
tween the latitude values computed for the north and south stars was 
greater than this value in each station computed and for each refraction 
model, 

If the difference or spread obtained between the north and south 
star latitudes can be used as a criteria, it can be seen that the 
Garfinkel Model gives a consistently larger difference, at the zenith 
distances observed (about 50 degrees), than either the Coast and 
Geodetic Survey Model or the Baldini Model. These latter two models 


give results very close to each other in all the stations computed, 


In a comparison of the refraction models at various temperatures 
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and pressures, which ranged to extremes of each quantity. It was found 
that the Coast and Geodetic Survey and the Baldini Models are consist- 
ently closer to each other in the refraction for a specified zenith 
distance, except for zenith distances over 80 degrees, than the results 
obtained from the Garfinkel Model. Since the Baldini Model was only 
designed for zonith distances up to 75 degrees, this seems to be a good 
comparison. Jt must be noted, however, that up to a zenith distance of 
50 degrees, the difference in the refractions given by the three models 
is less than 0.3 seconds of arc. The Garfinkel Model is the only one of 
the three that were investigated which is designed to give refraction 
for zenith distances of 90 degrees or larger. 

The Baldini Model was investigated for the effects of relative 
humidity and the wave length of the observed light. It was found that 
the humidity did not have a very significant effect on the refraction 
output of the model until a zenith distance of about 70 degrees was 
reached. The difference in wave lengths of the received light causes 
a greater variation, but its effect is diminished when it is considered 
that nearly all the observable stars give off light in the yellow wave 
length region of the spectrum. Consequently, the Paldini model was 
used with a relative humidity of 60 percent and a wave length of yel- 
low light equivalent to that used for the basic theory of the Coast 
and Geodetic Survey Model. The results for the Baldini Model would 
have been a bit more accurate if the relative humidity had been taken 
at the observing stations and the wave length of the observed light 
known, It is considered that,for the purposes of this project,these 
factors would not have had a very great influence, 


It was attempted to ascertain if an adjustment for the con- 





ditions at the observing station could be made to improve the refrac- 
tion .rodel, Sufficient data were not available to determine if this 
adjustment was feasible, It is suggested that this area could be one 
of future research but that data should be taken at a variety of zenith 
distances to determine if the adjustment method would actually improve 
the refraction results, 

Finally, it may be said that the three refraction models in- 
vestigated in this project are adequate for second order latitude de- 
termination employing the present two star observation methods; how- 
ever, it must be noted that none of these refraction models gives the 
ultimate solution to the refraction problen and, therefore, this area 


still remains open to future investigation. 


i, 











REFRACTION PROGRAM INFORMATION 


In all of the refraction subprograms used in this investigation 
and listed in this Appendix, the input and output values are in the same 
units. For the calling parameters the observed zenith distance is in 
radians, the temperature is in degrees centigrade, tho pressure is in 
inches of mercury and the relative humidity is given as a decimal frac- 
tion. All the subroutines give the output for refraction in seconds of 
arc. The requirements of each specific subroutine will now be listed, 

The Coast and Geodetic Survey Subroutine requires that the 
values of refraction from Table V of U. S, C.&£GS5S Special Publication 
a3/ (3) be read into the main program, Since these are listed for 
every ten minutes of arc this consists of reading in 541 values, At 
the same time a corresponding quantity in degrees is generated by the 
program, The Subroutine is called with the calling parameters in the 
following order: Relative Humidity, Temperature, Pressure, Zenith 
Distance, Literals statements are included in the subprogram to de- 
termine the factors for temperature and pressure conditions due to 
deviation from the standard conditions assumed for the computation of 
the tables, These values are from Tables VI and VII of the publication 
mentioned above, 

The Baldini Subroutine uses a literals statement to includo a 
table from which the water vapor pressure at the observing station may 
be computed, This is Table I of the Baldini ee The value 
"Slite" in the program is the wave length of light used in the computa- 
tions, This subprogram is called in the following order: Relative 


Humidity, Temperature, Pressure, Zenith Distance, 





All of the Garfinkel Subroutines listed in this Appondix have 
the same calling parameters. These are called in the following order: 
Zenith Distance, Pressure, Temperature, In all of the programs the 
gamma factor and other constants used are included through the use of a 
literals statement, 

The program for a comparison of the refraction models is fairly 
straightforward. It reads in the values required for the Coast and 
Geodetic Survey Subroutine and then proceeds to call each of the sub- 
routines given above to determine the refraction for various temperature 
and pressure combinations. Since these combinations range to extremos, 
the Coast and Geodetic Survey Subroutine had to be extended to include 
temperature and pressure correction factors for the entire range. This 
was done by simply expanding the literals statement for these factors 
to cover more range of the respective tables they were taken from, 

This expanded form is not shown here, 

The last item in this Appendix is a comparison of the various 
versions of the Garfinkel Subprogram. The Aberdeen and Coast and 
Geodetic Survey comparisons were made for the newer constants and a 
gamma factor of 8.7137. The other two comparisons used Garfinkel's 


original constants and a gamma factor of 8.1578. 





START 


F USREF 


TABLE 


PROGRAM STATEMENTS AND SCATRAN SUBROUTINE 
U. S. COAST AND soles aul REFRACTION MODEL 
DIMENSION (RM( 541), ZA( 541) )= 
READ IMPUT, USREF, ((RM(K),K=0,1,K.LE540) )- 
(12F5.1)@ 
DO THROUGH (TABLE),I=0,1,1.L. 540- 
DO THROUGH (TABLE), J=0,1,J.L.6- 
IJ = I + Je 
ZA(IJ) = (1/6.0) + (J * 0,.1666666667)-~ 
CONTINUE 
ZA( 540) = 90.0- 
SUBROUTINE (RCGS) = REFCGS, (HUM,TEMP,PRESS,ZRS)- 
DIMENSION (CENT(46),CT™M(46), BAR( 30) ,CBM( 30) )- 
UNIVERSAL (RM,ZA)- 


ZDR = ZRS « 57.2957797~ 


LITERALS (CENT, 5:0,5.6;6.1,6. 7, 7s20700 Oe gC. OL 1OMOR 
10.6, 11..1,412.7;1212,12.8,13.3,13.07 14 On ono 
16.751 7021 7.6410. 3,16.9,19 4.2020, 20 see tel. 7c 
22.8.23.3,23.9, 24.4, 25.0925.6, 20.1520. 7 ner een moreC. oF 


28.9,29.4, 30.0)~ 


LITERALS (CTM,1.018,1.016,1.014,1.012,1.010,1.008,1.006, 
1,.004,1.002,1.000,0.998,0.996,0.994, 0.992,0.990,0.988, 
0.986,0.985,0.983,0.981,0.979,0.977,0.975,0.973,0.972, 
0.970,0.968,0.966,0.964,0.962,0.961,0.959,0.957,0.955, 
0.953,0.952,0.9 50,0.948,0.946 ,0.945,0.943,0.941 ,0.939, 
0.938,0.936 70.934 )= 


LITERALS (BAR,28.1,28.2,28. 3, 28.4, 28. 5,28.6, 28.7, 28.8, 
28.9,29.0,29.1,29. 2 29 631294, 29.5429 .6,29.7,29.8,29.9, 
BOO, 0c 1s 30s es 00s De ed oUre 5, 30.6, 30.7 2055550. 9 1p = 
LITERALS (CEM,0.939,0.942,0.946,0.949,0.953,0.956,0.9599, 
0.963,0.966,0.970,0.973,0.976,0.979 ,0.983,0.986,0.989, 
0.992,0.996,0.999,1.003,1.007,1.010,1.013,1.016,1.020, 
1.023, 1.026, 1,029,1.033,1.036)- 





GEDT 


GO1 


GEDP 


GO2 


GEDZ 


FINAL 


CT="G@e 
DO THROUGH (GEDT), IG=0,1,I1G.L.45- 

JG=I1G+ 1- 

PROVIDED (TEMP.GE.CENT(IG).AND.TEMP.LE.CENT(JG)), 

CT = CIM(IG) + (CIM(JG) = CIM(IG)) * (TEMP - CENT(IG))/ 
((CENT( JG) - CENT(IG))- 

PROVIDED (CT.G.0),TRANSFER (GO1)- 

CONTINUE- 

CB = 0- 

DO THROUGH (GEDP), IB=0,1,1B.L.29- 

JBv= TB aol= 

PROVIDED (PRESS .GE.BAR(IB).AND.PRESS..LE.BAR(JB)), 

CB = CBi(IB) + (CBM(JB) - CBM(IB)) + (PRESS - 
BAR(IB))/( BAR(JB) - BAR(IB))- 

PROVIDED (CB.G.0.), TRANSFER (G02)- 

CONTI NUE 

RT = Ou 

DO THROUGH (GETZ), I=0,1,1.L.540~= 

J=I+ie- 


PROVIDED (ZDR.GE,.ZA(I).AND.ZDR.LE.ZA(J)), RT = RM(I) + 
(RM(J) = RM(I)) * (ZDR = ZA(I))/(ZA(J) - ZA(I))- 


PROVIDED (RT.G.O.), TRANSFER (FINAL)- 
CONTI NUE- 

RCGS = RT * CT * CB 

NORMAL EXIT- 

END SUBPROGRAM- 


END PROGRAM (START)-= 





VAPOR 


CONT 


SCATRAN SUBROUTINE 
FOR THE 
BALDINI REFRACTION MODEL 
SUBROUTINE (RB) = REFBAL, (RH,TDC,ATPRS,ZST)~- 
DIMENSION (HEAT(10),HG(10) )- 
PATM = ATPRS * 25,39998~ 


LITERALS (HEAT,-5.0,0.0,5.0,40.0)15-0- 20,0525, 0700n00 
35.0,40.0)- 


LITERALS ( HG,3.02,4. 58,6. 54,9.21,12.79,17.59,236 70. 
31.86,42,23, 55.40)- 


HGMM = O-= 
DO THROUGH (VAPOR), IF = 0,1,1F.L.9—= 
JF = IF + ie 


PROVIDED (TDC.GE.HEAT(IF).AND.TDC.LE.HEAT( JF)), HGMM= 
HG(IF) +(HG( JF) - HG(IF)) * (TDC - HEAT(IF))/5.0- 


PROVIDED (HGMM.G.0), TRANSFER (CONT)- 
CONTINUE = 


E = RH * HGMMe 


SLITE = 0,578- 
AO = 0.99827 
Al = 20"00150s 
A2 = 0,000006 


FNG = 2876.04 = 16,288/(SLITE.P.2) + 0.1367(SLITE.P.4)~— 
FNG = FNG * 1.0.X.-7-~ 


FNO = (FNG/(1.0 + 0.00367 * TDC)) * PTAM/760.0 - 
(5.5.X.-8) * E/(1.0 + 0.00367 * TDC)~- 


FNO = FNO * 206264, 81- 
TAN = SIN.(ZST)/COS,(ZST)~- 


RB = FNO * AO * TAN + FNO * Ai * (TAN.P.3) + FNO * A2 * 
(TAN.P.5)- 
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NORMAL EXT T~ 


END SUBPROGRAM-~ 











HALT 


GR15 


GRi2 


GR1i20 


SCATRAN SUBROUTINE 
FOR THE 
GARFINKEL REFRACTION MODEL 

ABERDEEN VERSION 
SUBROUTINE (R) = REFGAR. (ZOB,PP,1M)~= 
CALL SUBROUTINE ( ) = FPSET. (3,HALT)= 
CONTI NUE~ 
LITERALS (FN,4.256,FK, 4472.8, BE, 0.044385, GAMA, 8. 7137 )= 
UPPER COMMON (IND)-= 
BOOLEAN (IND)- 
DIMENSION (B(5))- 


PR = PP/760.0 * 25,3998- 


T =" 273.16 + 120< 


W = PR/T/T- 


TRANSFER (GR15,GR12,GRi2) PROVIDED (ZOB = 0.7853981634)- 


TANTA = SIN.(ZOB)/COS.(ZOB)) * SQRT.(T)/GAMA- 


THETA 


ATAN.( TANTA )— 
TRANSFER (GR120)-= 


TANTA = GAMA * (COS.(ZOB)/SIN.(ZOB) )/SQRT.(T)- 


THETA = ATAN,(1.0/TANTA)- 


PROVIDED (TANTA.E.O), THETA = ZOB- 


PS = 0,0- 
S = 0,0= 
SUM = 0,0- 


DO THROUGH (GR32), J=1,1,008221— 


Pie= 7 ais 
FJ = J ie 
FM = FN * (FI = Fd + 1.0) - FI- 
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GR100 


GR13 


GR1i31 


Vas ye 


GR1i4 


FJJ = Fd + 1.0- 
CALL SUBROUTINE ( ) = GAMMA, (FIJJ,FIJdJ)- 
PROVIDED (IND), TRANSFER (GR200)- 

FidJd = Fiat J 

CALL SUBROUTINE ( ) = GAMMA, (FIJJ,FIJJ)- 
PROVIDED (IND), TRANSFER (GR200)- 

FMS = FM + 1.0- 

CALL SUBROUTINE ( ) = GAMMA. (FM,FNG)- 
PROVIDED (IND), TRANSFER (GR200)- 

CALL SUBROUTINE ( ) = GAMMA. (FMS,FMS)- 
PROVIDED (IND), TRANSFER (GR200)- 

D = FMG * FMS- 

TRANSFER (GR13,GR31,GR13) PROVIDED (FM - S)- 
D = (FM + S)/(FM - S) * D- 


KK"="2,0"° "> + 1,.0= 


CH = COS,(THETA/2.0).P.KK- 
SH = SIN.( THETA/2.0).P.KK- 
TA = SH/CH- 


TRANSFER (GR132, GR131, GR131) PROVIDED (SH - 0.70710678)- 


CN = CH/SH- 
TA = 150/60 
Pi = FMG * FMG/D * TA- 


TRANSFER (GRi4, GR30, GR14) PROVIDED (FM - S - 1,0)- 
D = (FM + S + 1.0)/(FM - S = 1.0) * De 
KK + S * 2.0 + 3.0- 


ST = SIN.( THETE/2.0).P.KK- 


CT = COS.( THETE/2.0).P.KK-~ 
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TN = ST/CT- 


TRANSFER (GRi42, GRi41, GR1I41) PROVIDED (ST - 0.7.710678)~- 


GRi41 CN = CT/ST- 


TN = 1.0/CN- 
GR142 Pi + Pi + FMG * FNG/D * TN- 


GR143 PS = PS + Pie 


TRANSFER (GR1i6, GR31, GRi6) PROVIDED (P1)- 
GRi6 «= S=S + 2.0- 

TRANSFER (GR17, GR24, GRi7) PROVIDED (S - 2.0)- 
GRi7 TRANSFER (GR24, GR24, GR30) PROVIDED (S ~ FM)- 
GR24 P= Pi- 

TRANSFER (GR100)~- 
GR30° «=PS = Posse rs— 
GR31. -FMNIF = FM + FI- 

CALL SUBROUTINE ( ) = GAMMA, (FMNIF,FMNIF)- 


FMM 


i) 


(-1,0).P.(J-1) * FMNIF/(FJJ * FIJJ * FMG)~- 
SUM = SUM + FMM * PS- 
PS = 0,0- 

GR32 S = 0.0- 
B(I) = FK * BE.P.(I-1) * SUM- 

GR33. SUM = 0,0- 


R = SQRT.(T) * W * (B(1( + W * (B(2) + W * (B(3) + W * 
(B(4) + W * (B(5)))))- 


NORMAL EXIT~ 
GR200 CALL SUBROUTINE ( ) = ENDJOB. ( )- 
END SUBPROGRAM- 


SUBROUTINE ( ) = GAMMA, (Z,GZ)~- 


(aa 





rz} 


GAY 


GA10O 


GA11 
GAi2 


GA13 


GA14 


GA80 


F814 


GA9O 


F914 


LITERALS (C1,0.001917526918,C2,0.0008417508418, 
C3,0,00059 52381 , C4,0.00079 365079 36508,C5,0.0027777777777, 
C6 ,0. 08333333333,C7,0.9189 358533 )- 

UPPER COMMON (IND)- 

BOOLEAN (IND)- 

eee 

CV = 1.0- 

TRANSFER (GA10, GAi1, GA11) PROVIDED (X -10.0)- 

CV = CV * xX- 

X = X +1,0- 

TRANSFER (GA9)- 

TRANSFER (GA12, GA80, GA12) PROVIDED (CV)- 

TRANSFER (GAi0, GA13, GA13) PROVIDED (.ABS.CV - 1.0)- 
Mee WAV GK 


FX = (((((C1 * ZZ) + C2) * ZZ =~ C3) * ZZ + CH) * ZZ = C5) 
+7 eos 


FX = FK/X- 

G= (X - 0.5) * LN.(X) + C7 - X + FX- 

TRANSFER (GAi4, GA90, GA9O) PROVIDED (G - 700.0)- 
GZ = EXPE. (G)/cv- 

NORMAL EXT T- 

WRITE OUTPUT, F81, (Z)- 

(34H GAMMA OF ZERO OR NEGATIVE INTEGER, £14,8)- 
LNBC—jeOeeavarcr 77 ( (= 


NORMAL “XI T= 


WRITE OUTPUT, F91, (Z)- 


(44H GAMMA FUNCTION WILL EXCEED MACHINE CAPACITY, E24,8)- 
IND(=).0.777777777777- 


NORMAL EXT T= 
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BND SUBPROGRAM~ 











SCATRAN SUBROUTINE 
FOR THE 
GARFINKEL REFRACTION MODEL 
U. S, COAST AND GEODETIC SURVEY VERSION 
SUBROUTINE (R) = REFGAR. (ZZ,PR,TT)- 
FLOATING (Ni,N2,N3,N4)~ 


LITERALS (N1,1050,.61030, N2, 706.11502,N3, 262.0608, 
N4, 142.67293, GAMMA, 8.7137 )= 


fi 


TT/273.16 = 1.0~ 


P 


PR/760.0 * 25,39998~ 


w= F/Che 1) 


TANB = (SQRT.(T).GAMMA) * (SIN.(ZZ)/COS.(22Z))- 
B = ATAN.(TANB)- 

BH = B/2.0- 

TANBH = SIN.(BH)/COS.(BH)- 

FT = W * (SQRT.(T))- 


R= FT * ((N1 * TANBH) + (N2 * (TANBH.P.3)) + 
(N3 * (TANBH.P.5)) + (N4 * (TANBH.P.7)))- 


NORMAL EXIT~ 


END SUBPROGRAM- 
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SCATRAN SUBROUTINE 
FOR THE 

GARFINKEL REFRACTION MODEL 
ACIC FORM 


SUBROUTINE (REF) = REFGAR, (ZEN,PRS,TM)- 

DIMENSION (FN1(61 ),FN2( 50), FN3(15))- 

LIDERAUS (HNIE0.0,8.751763,25-9. 346043. 31 02.0,00. 7 co ee 
78.3,07<1.90.0,104.9, 113.8, 122.9, 151.9, 141.-0,150, 2,155 cr 


168.9,178.3,187,8,197.4,207.1,216.9,226.9,237.0,247.1, 
2574, 267.9,278, 5,289 2,300.1, 311.2, 322.4, 333.9, 345.9, 


357.3, 309.4, 381 079 394, 2,407.0, 420, 0,433. 3, 446 9,460.8, 
475.0,489 6, 504.55519 67, 53504, 551.55 50729, 584.9 602.4, 
620.3,638.8,657.8,677.4,697.7,718.6 )= 

LUTERAT Se (PN22620.0-050.0,0.1,0. 1.051021 0-1-0220 ace 
Oe s0e, Ons, O05 0.4 .0.5,026,0.7,02770.6,0.9, 1e0let ieee 
alee Saeed eel po srl + Dla Oe On Oreos De late eee 
Gey 9h 969 Oo Dy Cos (oO oOo 399609 e 7 sheds 11. tee. je 

LI TERAES WE n0.0),0.1,05 1,061,050, Orde Url OO conc. 
052405250. 3,0.3,0.4)= 


TS = 273.0/(TM + 273.0)- 

T = SQRT.(TS)- 

GAMMA = 8,1578- 

P = (PRS/760.0) * 25.39998- 
W=P* TS * TS- 

COTZ = COS.(ZEN)/SIN.(ZEN)- 
GAMO = GAMMA * T- 

TAN = 1,0/(GAMO * COTZ)- 

THETA = ATAN.(TAN)- 

ANG = THETA * 57.2957795- 

DO THROUGH (ONE), I=0,1,1.L.60- 
J=I1+ i- 

PROVIDED (ANG.L.I.OR.ANG.G.J.), TRANSFER (ONE)- 


Bi = FNi(I) + ((FNi(J) -FNi(I)) * (ANG - I))- 
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SETI 


SET2 


ONE 


OUT 


PROVIDED (I.LE.10), TRANSFER (SETI )—- 
IK +1 - ile 


JK 


J - 1i- 


B2 = FN2(IK) + ((FN2(JK) - FN2(IK)) * (ANG - I))-~- 


PROVIDED (I.LE.45), TRANSFER (SET2)- 

IM=I - 44- 

M= J = 46- 

B3 + FN3(IM) + ((FN3( UM) - FN3(IM)) * (ANG - I))- 
TRANSFER (OUT)- 

B2 = 0,0- 

B3 = 0,0- 

TRANSFER (OUT)- 

CONTI NUE~ 

REF = (W*B1)/T + (W * W * B2)/T + (W * W * W * B3)/T- 
NORMAL EXIT- 


END SUBPROGRAM~ 
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START 


F USREF 


TABLE 


SCATRAN PROGRAM 
FOR THE 
COMPARISON OF THE REFRACTION 

GIVEN BY THE THREE MODELS 

FOR VARIOUS TEMPERATURE 

AND PRESSURE COMBINATIONS 
DIMENSION (RM(541),ZA( 541) )~ 
READ IMPUT, USREF, ((RM(K), K=0,1,K.LE.540) )= 
(12F5.1)~- 
DO THROUGH (TABLE), I=0,1,I.L. 50~ 
DO THROUGH (TABLE), J=0,1.J.L.6— 
IJ = I + J 
ZA(IJ) = (1/6.0) + (J * 0.166666667)- 
CONTI NUE | 
ZA( 540) = 90.0- 
RHUM = 0.60- 
DIMENSION (CENT(3),PRSMM(4))- 
LITERALS (CENT,-5.0,10.0,35.0)= 
LITERALS (PRSMM,711.0,737.0, 760.0, 787.0)= 
DO THROUGH (END), I=0,1,I.L.3-= 
CTEM = CENT(I)- 
DO THROUGH (END), J=0,1,J.L.4~ 
WRITE OUTPUT, Fi- 
PRSIN = PRSMM (J)/25.39998— 
DO THROUGH (ONE), K=0,1,K.L.19- 
FACT = Ke 
ZD = 5.0 * FACT- 


ZRAD = ZD/57.295779 5- 
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ONE 


END 


Fl 


CALL SUBROUTINE (COAST) = REFCGS, (RHUM,CTEM,PRSIN, ZRAD)- 
CALL SUBROUTINE (BAID) = REFBAL. (RHUM,CTMM,PRSIN,ZRAD)- 
CALL SUBROUTINE (GARF) = REFGAR. (ZRAD,PRSIN,CTEM)- 
WRITE OUTPUT, F2, (ZD,COAST,BALD,GARF)- 

CONTI NUE- 

WRITE OUTPUT, F3, (CENT(I),PRSMM(J))- 

CONTI NUE- 

(1H1,Q$ COMPARISON OF REFRACTION FROM O TO 90 DEGREES AT 
FIVE DEGREE INTERVALS$//Q$ ZENITH DISTANCE $,5X,Q$ COAST 
AND GEODETIC SURVEY MODELS, 5X,Q$ BALDINI MODEL$, 5X, Q$ 
GARFINKEL MODEL$).- 
(//6X,F4.1,20X,F10.6,19X,F10.6,10X,F10.6)= 

(//Q$ ABOVE VALUES ARE IN SECONDS oF ARC$//Q$ THIS DATA 
WAS COMPUTED FOR A WAVE LENGTH OF 0.578 MICRONS$//Q$ 
RELATIVE HUMIDITY EQUALS 60 PER CENT$//Q$ CENTIGRADE 
TEMPERATURE EQUALS$,F7.1//Q$ ATMOSPHERIC PRESSURE EQUALS$, 
F7?.1,1X,Q$ millimeters$)- 

CALL SUBROUTINE ( ) = ENDJOB, ( )eq 


U. S. COAST AND GEODETIC SURVEY SUBROUTINE 
(MODIFIED FOR A LARGER RANGE OF TEMPERATURE AND PRESSURE) 


BALDINI SUBROUTINE 
GARFINKEL SUBROUTINE 


END PROGRAM (START)- 
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CCMPARISON OF THE GARFINKEL MODEL SUBROUTINES 


Coast and Aberdeen 
Zenith Goodotic Version 
Distance Aberdeen Survey Original ACIC 
degrees Version Version Constants Form 

0 0.000 0.000 0.000 0.000 

5 5090 5.088 Sales 5-157 

10 10.257 10.254 10.325 10.367 
15 15.587 15,582 15.690 15. 202 
20 as 21.164 AS 21.288 
25 27.120 he Ms a 27.300 27.266 
30 33.574 33. 564 33.796 33.784 
25 40.711 40 699 40.980 40.979 
4O 48.775 48.760 49.097 49.091 
45 bo. 107, 58.091 58.492 58.463 
50 69.216 69.198 69.674 69.685 
55 82.887 82.867 83.435 83.422 
60 100.414 100.393 101.079 101.059 
65 124.100 124,801 124,922 124,922 
70 158.465 158.457 159.418 159.554 
75 213.740 213.770 215.169 215.100 
80 318,633 318.825 320.830 320.933 
85 589.651 Belinea, 594.659 594.666 


Temperature = 10° Centegrade 
Pressure = 760.0 mm of Hg 


Refraction in Seconds of Are 
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ADJUSTMENT OF THe REFRACTION MODEL 
FOR STATION CONDITIONS 
This adjustment procedure makes use of the Gonoral Method of 
Adjustments to obtain a solution for the variation of the assumed 
10) 


arcane oie.s The general matrix form may be represented in matrix 


notation as 
a) a 


B! A | : A 
At oO | Ix | 0 


or may be given in a matrix equation as 


BF 


In these forma all letters represent matrices (with the exception of 
the subscript). The A, B and W matrices are defined as 


id 
So 


> 
3 


F(Ly XQ) 


where Ly, represents the observed quantities, L represents the adjusted 


A= 


Vv 


rs 


to 
H 
ie 


W 


values of the observed quantities, X, represents the assumod parameters 
and X, represents the adjusted values for the parameters, Other matrices 
used in the solution of the equations are the V matrix which gives the 
residuals for the observation, the X matrix which gives the corrections 
to the assumed paramoters and the P matrix which is the weight matrix, 
The prime symbols on the matrix indicate that it is the transpose of the 
matrix whose letter is used and the minus one notation indicates that 
this matrix is the inverse of the matrix whose letter is used, Dc- 


fining a new matrix as 
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M = BP7 Bt 


tho final form of the equation for the X matrix is given as 
X = -(am7tay7 Osea 

For an adjustment to take place the condition must bo satisfied 

that 
N+ US pes u 

where n is the number of observations of the observed quantities, u is 
the number of unknown parameters and r is the number of condition 
equations. When these conditions are fulfilled then the adjustment 
can take pleeen 10) 

The above factors and principles are thon an outline of the 
method which was used to attempt to adjust the refraction coefficients 
for the conditions at the stations, 

Two mathematical structures were employed in the adjustment 
attempt with two seperate attempts being used to make the adjustment, 
In the first instance the observed quantities were the observed al- 
titude and hour angle of each star, This gave four observed quantities 
per condition, In the second instance only two observed quantities 
were required, these being the altitude and azimuth for the single star 
inquestion. In both cases the parameters were the three refraction 
model coefficients and tho assumed latitude. The declination, tempora- 
ture, pressure and time botween direct and reverse sightings of the 
instrument were considered not to vary for the purposes of the adjustment, 

The equation used for the rofraction model was based on the 
ACIC form of the Garfinkel Iiodel which is given here as 


R= YT) WN,K cot h + NK? coth. + Nak? cot7h ) 


where 
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Woo P/ it 


sl 
o/¥ 
with Y assigned the Aberdeen value of 8.7137. In this form P and 


i 


K 


Ty are the relative pressures and temperatures, Those quantities 


arc obtained from the relation 


T =T + 273,16 
0 O 
20a 


Pa=Oe/ 760.0 
where the observed temperature is in degrces centigrade and the 
observed prossure is in millimeters of mercury, Tho cotanjent funce- 
tion appears in the equation since 


tane = T, tan 2,= Ty cot h, 
¥ cd 


where Z, is the observed zenith distance and h, is the observed alti- 
tude, 

The relation may also be established that 

hi =e it 
where h is the observed altitude after correction for the refraction 
and R is the refraction correction from the refraction model. These 
relationships will be used in the normal equation coefficients which 
are derived for each mathematical structure, 

The first mathematical structure investigated was based on the 
fact that,if all things were perfect, the latitude obtained from an 
observation of a south circummeridian star would be tho samo as that 
obtained from the observation of Polaris. This is fitting the more 
genoral statement that tho latitude obicined from the north star should 


be the same as the latitude obtained from the south star to our avail- 
able data, The basic condition is then that the difference of these 


two values be equal to zero, This mathamatical structure is given as 
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4 
( 
= ; fan O 


DI — 


rae GE iw 
~ 


ry 


with the subscripts denoting the north and south stars, The reduction 
methods used by Thorson‘? ? were used to obtain these values in terms 
of the observed values and parameters, Using Polaris, the latitude 
from the north star observation becomes 
oN = Dewar eCOs ete 
where 
P= (90 ~S) 
B= 2F* tanh sint 
with t representing the hour angle, §& representing the declination and 
h being the corrected altitude of Polaris, The latitude from the 
circurnmeridian star may be represented as 
Cres nis OG. 
where 
m= hh + Am 
m= my +m = 0,0001364(AT°)* + sin@3 


A =cos 5 cos} 
cos h 


with h representing the corrected altitude, t given as the hour angle 
of the south star, S representing the declination and A T° given as 
the time in seconds between the direct and reverse pointings of the 
instrument. Now denoting the north star quantities with an asterisk 
and the south star quantities with no superscript the mathematical 
structure may be written as 


pe 
h* —~ R* - P cos t* + aP tan (h¥ ~ R*) sin“t* 
0 ~S+h -R + cos Scos 0.0001364(n.7TS)2 + 2 sinest = 0, 
-90 Geo Une) 


90 





Since eight observations are taken of the north and south stars 
individually, this mathematical structure gives eight conditions to 
be fulfilled. There are four assumed parameters and a total of thirty- 
two observed quantities. This gives the values 

m 2 2 

r= 8 

ums G 
which when used to ascertain if an adjustment will take place gives 
the relation 

(32 + 4) >8>4. 

Therefore, an adjustment can exist for this amount of data used in the 
above mathematical structure. 

The A and B matrices are the ones whose form we are concerned 
with here. All abservations are taken to be of the same weight which 
causes the P matrix to be a unit matrix which may be dropped from the 
equations. It should be noted here that the values of the altitude and 
hour angle are correlated and that possible weights should be assigned 


to take account of this fact. The A matrix takes the fomn 








SF \_F Nae K :| 


and the B matrix is of the fomn 


mar SF x8 : 
Se * 
we = dh, at, 
For simplirication in the writing of the formulas for these partial 


derivatives some factors will be designated for use in the formulas. 


These factors are 


FR s Ny a aones + 3N4 ges cot? oO + 5N3 i \5 cot¢h, 


\ sinthg Sin sin2h, or SinZho 


=) 





FN = Ne cot Ao 
\: 
PA = COs S cos gisin (nos De 


cos“(h — R) 


FPOL = }?~ sin“t* 
Ce Chere) 
epBere Goya ts, 


cos iis R) 


FCIR 


tI 


which will be used along with the factors A and m defined abovo, 


The partial derivitaves for the A and B matricos may now be written 


ur 1 + FWCFR*) + FPOL [1 + FuCPR*) | 


3- 
= 





a = P sin t* + p* tan (Ha R*) sin L* cos*c* 

est orem «Beg ay [1 + ran] 

e+ as th ay 

a COs COSI (2 Sin st Cosest) 

dt cos oe R) 

Ve = —FW(FN*) — FW(FN*)FPOL- F(FN)(Am)FCIR 

ao = =-FW(OFN*)3—F(FN*)2 FPOL— Fa(FN)?— FW(FN)9(Am)FCIR 
2 

5 = -FN(FN*)9 — FW(FN*)? FPOL= Fw(FN)9— Fw(FN)?(Am)FCIR 
5 

3 r =- —Anl tan p 

Awe; 


where ie values with the askorisk once again represent values computed 
from the north star and those same values without a superscript represent 
these quantities computed from the south star. These derivitives are 
rather involved and a simplor systom would be more desirable if it 
could be found, 

A simpler and more general method of expressing the mathematical 


structure is to use the first equation for the astronomical trianglo 
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given in Section 4.1, This gives the form 

sin sin § + cos g cos S cos t- sinh = 0 
where the quantities used in the formula are the same as those which 
were described above, In this method, each star observation must ful- 
fill the condition given by the mathomatical structuro. 

For this method thore are thirty two observed quantities, six 
teen conditions to be fulfilled and four unknowns, The data used for 
this method was the samo as in the case of the first mathomatical 
structure given above, The assumed parameters remain the same in 
this adjustment and thorefore there is no change in the general form 
of the A matrix, The B matrix is reduced to two terms since only the 
observation of a single star is considered for each condition, This 
lets the B matrix assume the form 


| SF Sp 
mL aS 


where t is now the hour angle of the observed star and ho the observed 
altitude with no distinction being made for the north and south stars, 
For this method the values 

N= 32 
16 


ag 
u=4 
are obtained which give the relation 
(32564) S4e6as 4, 
This relation shows that an adjustmont can oxist for this mathomatical 
structure with the available data, 
The partial dorivitavos for this mothod are easier to work with 
than those which were taken from the previous mathematical structure, 


Using the constant factors Fv, Fil and FR from abovo the partial 


Ae, 





derivatives becono 


woo = — cos ¢ COs Bon © 
st 


ays Cos (tae R) [4 + aC FR) | 





Si ~cos(h,— R) (Fw) (FW) 

Se — R) (Fa) (FN)? 

Si cos(h R) (Fd) (FN) 

dF = —cos(h — R) (FN) (FN)? 

SN, Oo 

ee 2 cos sinS — sin cos § cos t 
oe of 4 ) OS 


with theso values being computed for each observed star, 

In both of theso adjustment methods given above the W matrix 
can be computed from the mathematical structuro, Then with tho A, B 
and W matrices the matrix equations can bo used for each adjustment and 
2 solution should be obtained if tho data are not too strongly 
correlated, 


The valucs for the refraction model coefficients were selected 


as 
Ny = 522.5 
Ny, = 86.9 
N, J ry 


from an examination of tho Garfinkel modol output for zenith distances 
from 40 to 60 degrees, Tho numorical values given above aro an average 


of all the coefficient values obtained by this process, 
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